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V .ABSTRACT ^ ' • * 

These proceedings of a conference on 1978 youth 
knowledge development activities implemented under the Youth 

. Employment and Demonstration Projects Act (YEDPA) of 1977 consist of 
the proiseedings of the ^ive sessions of the conference and the texts 
of 16 conference papers. Discussed in the first segtion of the 
proceedings arie demonstrations and research conducted under, the . 

* auspices of the Youth Incentive Entitlement Pilot Projects (YIEPP) ^ 
the yiruth Community Conservation and Improvement Pro jects- (YCCIP) , 
and the Youth Employment and Training Programs . (YfiTP) ; basic 
research; and data collection and evaluation. Topics covered ^n the 
conference reports include the effectiveness of various research and 
job training ptograms, YEDPA researc^h and interagency coo'perati^on and 
discord, ah economic analysis of the patterns and l^rends in youth 
unemployment, the potential impact 6f employment and job training 
programs on youth unemployment, the social and economic significance 
of teenage tinemploynent, youth experiences as^pathways into the world 
of work, and the economic impact of the Job Corps. (Belated youth 
knowledge and development reports are available separately through 
a IC — see note.r (HN) 
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OVERVIEW 



Congress and tiya Adminis^tration have made a major commit-^ 
irient to expanding and improving employment, training, and * 
career development services for youth. On August 5, 1977, 
the President signed the Youth EmpJoyijient and Qemonstration 
Projects Act (YEDPA) which created for new programs/arid pro- 
vided significant discretionary authority to test new approaches 
for aiding youth. ^The new programs were subsequently extended 
through fiscal 1980 and backed up sizeable appropriatidrls . 
In addition fo these new efforts, the Job Corps program which • 
provides comprehensive services in^'a residential setting for . 
severely disadvantaged ^uth has been doubled in size to 
44,000 slots. Finally^ the Summer Youth Employment Program 
(SYEP) has been reassessed and refocused with the goal of sub- 
stantially increasing Its effectiveness ,r ^ 

Knowledge development activities are a critical element in 
these youth Initiatives.- ' Resear'ch, -evaLuatiorL-and.-demonr™. 
stration efforts reldted- to youth were a major' priority in 
the 1960 's 'but have received declining attention in recent 
years. Whil^ muidh was learned, many critical questions re- 
main unanswered about the scale and nature of the problems 
and the effectiveness of public interventions. A primary 
aim of YEDPA was, therefore, to improve the information base 
for public policymaking. It was designed as a*limited dura- 
tion "demonstration act" which mandated a range of experi- 
mental, research and evaluation. efforts. Other longstanding 
youth programs shared this emphasis: 'Job Corps had tra- 
ditionally been a labora^tory 'for experimenting with new, 
approaches to serve economically disadvantaged youth. With 
expansion, there was a need for innovative approaches as well 
as improved ^assessment of current operations. Basic infor- 
mation was also lacking about SYfeP and possible improvements. 
Congress and the Administration wanted to know mor^befor^ 
developing and implementing youth policies, for the, longer 
term. - ^ ' 

An Office of- Youth Programs was established in the Employ- 
ment and Training Administration in July 1977 to implement 
these youth initiatives. A major dimension of its missioipf 
was to develop and coordinate knowledge development activ- - . 
ities for YEDPA, Job Corps .and' SYEP while serving- as a nexus^ '\ 
for linking with youth-related efforts of oth^er agencies. 

.In fiscal 1978, an ambitious' agenda of demonstration, research 
and assessment activities was implemented. A Knowledge Develop- 
ment Plan for the Youth Employment and Demonstration Projects 
Act structured an array of JiEDPA discretionary efforts, which 
would address the most basic questions of concern to national ' 
policymakers. These efforts included a massive experiment with 
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job guarantee! as well as /almost '$100 million' in action pro- 
grams designed to yield information about what works best 
far youth. Demons tiration programs were mounted as part of 
' th^ Job Corps expansion effojrt^ and previously commissioned 
^research and evaluation work was completed. Research ahd 

evaluation related to the 1978 summer program was signif- ^. 
icantly broadened. With the extatision of YEDPA programs 
through 1980 , Knowledge development^ plans were also developed 
for fiscal 1979 a^d 1980. 

^The knowledge development activities for the three years 
represent an aggregate of more than half a billion dollars. 
While most of these resqurces are used to provide quality 
.services and ^mplqyment opportunities to youth, and while 

^they also ,are a/.me^hanism for institutional change as well 
as for achieving equity, they have been carefully structured 

~^ ' "SO""HS^ to -y^iel d-much^^mora-Jbhan, the usual information about 

what works for whom. Research, assessment and technical 
assistance components which .do not yi^ld direct " benefits 
to youth amount: to less than 2 percerit of total youth 
resources', yet with maximum structuring of action programs, 
this investment may yield substantial long-run knowlec^^e^ 
benefits. 

The speoific demonstration programs, the research work,_ and 
the evaluations are designed to ^ address 15 basic questions, , 
'which were specified in the legislation or were discussed in 
the debate over future youth .employment policies: ^ 

r 

> J, 

1. D6es school retention and completion increase the 
future employabiXity of potential dropouts and ^ 

th6 disadvantaged, and are employment and training / 
services linked to education: an effective mechanism \ 
for increasing s,chool tfetention and completion? 

2. ^ Can the school-to-work transition process be improved? 

This involve? several related questions. Are new 
Vinstitutional arrangements ^'feasible and warranted? ^ 
Will incifeased labor market information and assistance 
expedite the transition? Can new transition routes 
be created? 

3. '' Given the fact that work experience has becpme the 
/ i pr'imary emphasis of. youth progralns, are the jobs 

/' ^productive, -which ones are most "meaningful" and how ^ 
can. they be improved? 
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Doe's structured, disciplined worJc experience have as 
much or more impact on luture employability than other 
^ human resource development services or a combination 

of services and employment^ i.^.^ should public poli'cy 
* ^ ^ emphasize' straight work experience^ combir«r€ions of- 

^ work and training and other serviaes, or should training / 
education^ and supportive services be emphasized? 

5. Are there better approaches and delivery mechanisms 
for the types of caresi: development^ -employment— and 

^ * training services which are currently being, offered? 

6. To what, extent are 'short-run interventions and oijrtcomes 
Wela<ted to longer-term impacts on employability during 

adulthood? Put in another way, how much can public 
interventions redirect the developmental process? 

. — — What-iverks-"bes-t-for-whom? — WhcrL. peiloiiiicinyre or ouLcome^ 
standards are t>est to_ determine what does ^nd does not 
work -for youth? Which youth with what characteristics 
benefit from which programs' and approaches? 

8^ What is the universe of need for youtn\p'rograms? What 
' * is the cost of fu3»ly employing youth? How many would^ 
• , take jobs if %iiey were available and how many* hours of 
^ employment dq^ they require? 

9* <^hat approaches and procedures . can bemused to involve ^ . 

the private sector in employment ari'd training efforts 
and to Increase the placement o;E the participants in 
private sector jobs? How effective are these approachef^ 
' accessing new jobs and providing bettef career tracks 

/tor youth? Are they .pjrefferable to, public sector approaches? 

10. )Wl%A^t Is the, best mix of enrollees in terms of age and 
' . income status? Will poor youth benefit from interaction 
with nondisadvaritaged youth or with older persons? Is 
targeting . achieved and is it a worthwhile notion? ^ \ 



11. What arrangements can^be made to increase the duration 
of employment and ;^rainin9^ interventions and ^to assure 
that participants realize lifetime benefits?-^ Will youth 
demonstrate the commitment and consistency to make these 
lorig'-term investments pay;.off? ' ^ ^ ' 
♦ 

12/ What' strategies are most important at different points 
. in. the lives of youth? Must training be delayed until 
greater maturity is achieved? * Are employment and train- 
ing programs a way of inducing matXirity-? . ^ 
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13. Hov can separate youth programs be better integrated to 
improve administration -and to provide more comprehensive 
services to youth? To what extent, are the programs already 
j|.ritegrated at the . local .^.eve 12 ^ * * ^ 

14. How do the problems of significant youth segments differ 
including those of migrants / rural youth, the Xhandidapped; 
offenders, young women with ohildren, runaways and the 
like? Are special needs groups and special problems 
better handled by mainstreaming or be separate programs 
for these groups? 



15. How can the lessons' from knowledge development activities 
^ best be transferred to improve existing youth programs? 
How can the institutional change process be pr^oted? 
What are the learning curves on new programs and how 
much can they be expected to improve witK.time? 

These broad questions subsume many more. They will not be re- 



of a synthesis of a range of coordinated efforts /' 
including basic research, ^process evaluations, impact assess- 
ments of regular program long-term followup,^ structured experi- 
ments, demonstrations testing the feasibility of conc'eptSv^and 
better identification of currently existing model programs^nd 
components. ^ — 

The scale and scope of knowledge development activities are ^ 
staggering. Each demonstration, research and evaluation proDect 
had t6 be .separately and carefully designed m very shoifE^ 
order to get the activities underway. The clock^was running 
in the sense that YEDPA was initially for one year only and 
even with its extension through 1980, results were needed 
immediately to redesign youth policies for the 1980s. 

Given the scale, complexity and haste, it is important that' 
..the elements of this knowledge development ^effctt be contin- 
uously examined, that coordination be achieved wherever 
possible, \and the results be synthesized as they become avail- 
able.' Constant improvements and ^adjustments are requi'red in ^ • 
research desi gns and it is necessary to assure th^t the re- 
sults will be available in as timely a fashion as is feasible * 
to. address^ critical policy issj^es. ' ' 

To ^help assure these results, the Office of Youth^ Programs 
commissioned the Brandeis University Center for Piablic Servifce 
to organize a Conference of 19-78 Youth Knpwledge Development 
Activities which would bring together the researchers on 
the major projects and many of the key players ultimately 
involved in translating the findings int© policy. The two- ' < 
day conference was held October 5 and 6, 1978. Short tack- 
ground papers were prepared with little lead timep'but the 
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ma j or ^purpose was to lay the ^^conceptual '^groundwork for^ a con- ' 

• tinuing assessment and modification process. The proceedings 
of a rather' lively conference have been siibstantially edited 
to focus on the major issues • The background' papers are in- 

* eluded with little pretense that they reflect any rigorous 
analyses • 

The results.. ^ar_e_ important^ in^several-^ays : -Pi3?str/— thereH:s — 
a good deal of basic. information about the projects and the 
related research. Second, the disc;ussions suggest ways in 
which the^ projects are -interrelated. Third, the needs for 
greater coordination and more careful "time-sequencing of re- 
sults are made apparent. Fourth, the complexity of demonstra- 
tion, evaluation and research in this area becomes clear. 
Fifth i the varying perspectives within the research community 

problems. Sixth, a realistic sense. of the limitations on re- 
search and evaluation activity is • demonstrated. Seventh, 
areas where more work is needed are identified. Eighth, the 
interrelationship between information, knowledge and policy 
is Jiighlighted. 

This volume will be useful for those seeking to get some under- 
standing of the knowledge development activities beyond th.e'^ 
specifics in the knowledge development plans- for 1978-198*0. 
The volume is also a foundation for continuing asse^isment and 
modification of these youth demonstration, evaluation and re- 
search activities. Never before^ hafs. such a- large scale, 
diversified set of activities been updartaken to address a^^ 
single social problem. The resijXtg' will yield much know- | 
ledge not only about youth ^n^loy me nt problems and policies, 
but also 'about the potentials of research, assessment and 
policymaking in our pomplex society. A followup conference 
in Jun-e 1980, Making Sense of the Knowledge Developnient Find- 
ings , as^esse^' the progress over the year ctnd a half since 
this initial meeting',' as well as' setting the stage for con- 
tinuing work on knowledge development. Howeyer, the first 
volume is important for more than historical reasons because 
it raises many of the key issuer which must be considered in 

dealing with the research, evaluation and demonstration^ 

findings. ^ ' . * 

■ ■ ■ 

This valume is just one of- the products ov the "knowledge 
development" effort implemented under the mandate of the 
Youth Employment and Demonstration Projects "Act" of 1977. 
The knowledge development effort consists of hundreds of 
separate research, evaluation and demonstration activities 
which will result in literally thousands of written products. 
The activities have been structured from the outset so that 
each is self-standing but also interrelated with a host of 
other activities, ^e framework is presented in' A Knowledge 
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Development Plan for the Youth Employment and Demons trat ion 
Projects Act of ,1977 ^ A Knowledge Development Pl^n for the 
Youth Initiatives Fiscal 1979 and Completing the Youth Agenda ; 
A Plan for Knowledge Development ^ Dissemination and Application 
in Figcal 1980 , " 

/ 

Information is available or will be coming available from the 
various knowledge, {development activities to help resolve an 
almost limitless^-^ajrray of issues, but answers to policy 
ques^tions will usually require, integration and synthesis 
from a number of separate products, which, in .turn, will de- 
pend on knowledge' and availability, of these products. A ^ 
major shortcoming of past research, evalXiation an^3 demons tra- , 
tion activity has been the failure to organize and disseminate^^^^ 
the products adequately to assure the full "exploitation of the 
findings. The magnitude and structure of the youth, knbwledge • 
developme nt effort puts a premiiun _o iLjirxian±2at ion-^ai^-^^ — - 

ination of findings. ^'^""""^^--^ ^ * ^ 

As part *of its knowLedge-development mandate, therefore, the 
Offipe of Youth Programs of the Department of Labor will 
organise, publish and disseminate the written products of . 
all major research, evaluation and demonstration activities 
supported directly by or n;ounted in conjunction with the 
knowledge 'development effort. Some of the same products * ^ 

may also be published and disseminated through other channels, 
but they will be included in the structured series of Youth 
Knowledge Development Reports in order to facilitat-e access 
and integration. ^ 

'The Youth Knowledge Development Reports , of which this is one, 
are divided into twelve broad categorie^s : 

* « 

1, Knowledge Development Framework^ The. products in 

. this category are concerned with the structure of knowledge 
development activities, the assessment methodologies which 
are employed, validation of measurement instruments, the 
tra'nslation of knowledge into policy, and the strategy for 
disseminating findings. " ' i 

2 , Research on- Youth Employment and 'Emplo yabili ty Develop- 
ment ; The products in this category represent analyses of 
existing data, presentati'on of findings from new data sources, 
special studies on dimensions of youth labor market problems 
and policy analyses, 

3, Program Evaluations ; The products in this category 
include impact, process and benef it--cost evaluations of youth 
programs including the Summer Youth Employment Program, Job 
Corps, the Young Adult Conservation Corps, Yduth Employment 
and Training Programs, Youth Community Conservation and 
Improvement Pro'jects, and the Targeted Jobs Tax Credit. 
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4, Service and Participant Mix ; The evaluations artd d^mon- 
strationi summarized in this category concern the matching of ^ 
different types o.f youth with. different service combinations. 
. This involves Experiments with work vs. work- plus remediation 
^vs. straight remediation as treatment options. It also includes 
attempts to mix disadvantaged and more affluent participants, as - 
well as ycAith "wi£h oldep workers. 

. *^ 

5 Edu cation and Training Approaches ; The products in this 
' category present the findings of structured experiments to test 
the impact and effectiveness, of varioils education and vocational 
training approaches 'including -specif ic education methodologies 
for the disadvantaged, alterna^i/ve education .and advanced 
career training,. / ^ 

^ 6. Pre -Employment and Transition Services: The products in 

.. .^-t-his-categox^pr^esent^he-f-i-nding& ot structured experiments to 

- test the impact and effectivWess of school-to-work transition 
activities, vocational exploration, job-search assistance and 
other, efforts to better prepare youth for. labor market success. 

7^ Youlh Wo rk Experience ; The products in this- category 
address the organization of work -activities, their output, pro- 
ductive roles. for youth and the impacts bf various employment 
approac^hes. . . ■ 

8. Implementation Issues ; This category includes cross- 
' cutting analyses of the practical lessons concerning "how-to- 

do-it'. " r^ue.s such as learning curves, replication processes 
and programmatic "batting averages" will be a.ddressed under 
■ " this categp^y, as well as the comparative advantages of alterna- 
tive delivery agents, -r- ' , - 

9. ' Design and Organizational Alternatives ; The products, 
in this category represent assessment's of demonstrations of 
alternative' program and delivery arrangements such^as consoli- 
dation, year-round preparation for summer _ programming, the use 
of incentives and multi-year tracking of individuals. 

^10. Special Needs Groups ; The products in this category *j 
present findings on the special problems of and adaptations ■ 
needed for significant segment-s including •minorities, young 
. mothers, troubled youth, Indochinese refugees and the handi- 
capped. . 

XI, Innovative Approaches ; The products in this category 
present the findings of those activities designed to explore 
new approaches. The subjects covered include the Youth. _ _ . 
Incentive Entitlement Pilot Projects, private sector initi- 
-atives, the national youth service experiment, and energy 
initiatives in .weatherization, low^head hydroelectric dam 
■ restoration, windpower and the like. 
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12. Institutional Linkages ; The products in this category 
will include studies of institutional arrangements- and linkages 
as well ^ as assessments of demonstration activities to encourage 
such linkages with education, volunteer groups, drug abuse 
agencies and t^i^like. 

In each of these knowledge .development categories, there will 
be a range of discrete demonstration, research and evaluation 
activities, • focused on different policy, progtam and analytical 
issues. For instance, all experimental demonstiratioh projects 
have- both process and impact evaluations, frequently undertaken 
by different evaluation agents.- Findings will be published as 
they become available so that there will usually be a series 
of reports as evidence accumulates J To organize these pro- . 
ducts, each publication is classified in one of the twelve 
broad knowledge development categories, described in terms of 
the more specific issue, activity or cluster of activities to 
which it is addressed, with an identifier of the product and 
what it"^^resents relative to other products in the demon- 
stra^ioji. /Hence, the multiple products under a knowledge 
development activity are closely interrelated and the activ- 
ities in each broad cluster have significant interconnections. 

This volume is closely interrela,ted with all tha other products . 
in the "knowledge development framework" category. ^ In partic- 
ular, it should be read in conjunction with the report on the 
follow-up conference; Makincf Sense of the Knowledge ^Development 
Fj.ndings as well as the basic conceptual ' outlines in The Know- 
l edge Development Agenda and the Office of Youth Programs 
synthesis of findings in Youth Knowledge Development Summary > 



ROBERT TAGGART 

Administrator 

Office of Youth Programs 
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INTRODUCTORY SESSION 



^ . ; • ■ ■ 

M R . TAGG ART : ' The 6ffice of Youth .Programs has beerr looking forward 
to this, conference with great expectation and also some trepidation. 
We have been busy in the last year and a half, distributing money, 
develop jlng grants, implementing research desig.ns, moving enormous 
mo.u-nt^ins of paper. - Vie relish this notion, of getting everybody, 
tpgether, taking stock of v;hat we've done and gettitig advice from 
some of the best -expert-s in the country. 

w ^ ' 

^ ■ ' On the other hand, we're also lookiijg to this conference with 
some degre'e of anxiety. The anxiety comes from the fact that w$ 
are opening up your research efforts at ^ very i4arly stage, looking 
at ^what's ongding instead Of wai^ting ufitifl the results are completed. 
The volume of activity is so great that v/e are not sure of the 

" quality of what Ve have ongoing,. Our effort has b^en concentrated 
on developing research and demonstration projects rather than » 
moni'^oririg , coordinating and integrating them. ^ ^ 

' The reason we in the research, evaluation and 'policymaking 
business are here is because Congress has challenged th'e Adminis-- 
• tration and the research community to com'e up with answers abo^t 
how better to serve them, .and, for the first time, h^is providdd" 
the resources and the flexibility to meet that mandate. . • - * 
' V 

The Youth Employment Demonstr^atipn Project Act^was signed 
by President Carter on ^^ugust 5, 1^77^— an Act which was initially 
for one year and nqw is going to be extended two more years — 
,,but one which was definitely a ^short-term demonstration A^Tt;. It 
provides massive demonstration authority, and gives specific 
research, demonstration, and evaluation mandates. 

First, it 'create.s what" is, I helieVe, thfe lafgesi: structured' 
. social experiment in our history: Youth Incentive Entitlement; ^ 
Pilot Pro^BCts v;hich will eventually amount to about $300 million 
in funding over three ye^rs. 

Secondly, under the Youths Cominunity Conservation and Improvement 
Projects portion of the bill, it gives discretionary authority to 
test the "sweat theory" ,- -that if you put kids to hard work ui^der 
careful Supervision, wLth a stress on output, it will have an 
impact on their life. * * ^ . 

Un^er another por1:ion of the. bill. Youth Employment and 
Training Programs, <^hich will fund any of a broad range o? approaches 
the Secretary of Labor is' given a large amount of diSfcretionary 
authority to- test everything under the sun^ for youth. But this 
is not all. At the same.tip\fe, we're doublings the size of Job 
Corps. With the resources that are provided for this expansion, 
there is the opportunity to use 'Job Corps, once again as a laboratory. 
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In Job Corps / you have more control, better records And more 
tracking than in any pther network. It is the only program where 
we have an intensive investment in human resources for youth. So, 
we have possibilities there as well as under YEDPA. • 

\ ' 

, At the same time, it was clear that Congpess was no longer 
satisfied with our knowledge of the Sununer Pr/ogram for Economically 
Disadvantaged Youth (SPEDY) . We were reporting back year after 
year about how m^n^ j^outh we' served and not about tl^e 'impacts or 
content >of services-. Congress wanted to know wKat was happening, 
whether the participants were going back to^ school, whether their 
summer jobs we!re good and whether they were supervise(^. ^ > 

Congress is asking us to 'come up with answer^ to. questions ' ^ 
about SPEDY, •^ajDout Job Corps, and about the ^EDPA programs. 
Accompanying these'' mandates and requirements Is a great deal of 
money. 

In pursuit of this ^ legislative 'mandate , we have been supported 
"by substantial, commitment within the Administration from the highest 
levels. When we first came into this effort, almost a year and a 
half ago, there was a meeting at .the t^hite House in which all the 
federal* agencies were carlled tcJgether and were told to cooperate, 

*'to share resources, ideas and responsibilities in a coordinated^ef f ort 
I presume that happens in every administration, but in this case * . 
there was follow-through, . containing commitments, and results. , 
P^i^iaps it means very little tc^out^iders , but we have some 17 
in^jlTagency agreements under the youth initiatives\ The Department 

^of C^bor^has worked cooperatively with almost every federal agency. 
Ther,e*^ a Vice President's Task Force on Touth Employment that's 
been established to i^rther this coordination, as well as ef foists 
withift the White House* There* is going to'be a. Presidential Policy 

.Review Memorandum,^ There is a very 'conscientious effort from the 
top to pyll this thing together and to l&5k at the research and 
demonstration issues and the results as 'an intergral part of the 
policy "process. 

Under YEDPA, the Labor Department had t^ie Jead action. We 
are lucky to. have a Secretary and Assistant Secretary who, ^re ^ 
very .much committed to doing what is necessary in order to learn 
what works and what doesn't.. They have stood by that commitment. 
For instance, they have stood by it when we were choosing Entitlement 
sites by a competitive process which provided as ^much as $20"millibn, 
to"^ a sing^le area and where political pressures were simply enormous. 
We had a Secretary and Assistant Secretary who didn * t waiver on the 
3eleet_j,dn on the bas^s of objective criteria. 

That's yery important in mounting any type of knowledge 
development effort., ^t the same time you had a" Secretary who 
ha^ been willing' to go forward with attempts at wage subsidy , » 
experiments, knowing there were going to be reservations on the 
part of the labor movement. He/ stood hf the experimental effort 
because ^he believed we have to learn what works and what doesn't. 
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An office of Youth Programs (OYP) was established in the 
Employment and Training Administration specifically to emphasize . 
research, demonstration^ and evaluation activities, as well as to 
coordinate and improve youth programs across the board. Early in 
the game,, we in OYP developed a Knowledge Development* Plan that 
outlined all the efforts that we would launch and all the' issues 
we wou-ld address. This plan was approved and every element has 
been implemented. I doubt that ever- before discretionary 
activity has been planned so carefully or delivered so completely. 
It's a testament to the support from the Secretary and the 
Assistant Secretary for' employment and training. 

You will be examining all the major pieces of 1978 discretionary , 
activity. There is now; a 1979 plan for another $215 -million or so 
coming down the pike. So we are talking about almost half a billion 
dollars in-^ the* discretionary projects related to youth, all of which 
have been structured .to learn more about whalj works and what doesnVt. 
It's a massive array of research, evaluation, and demonstration. 

What we will be reviewing in.theViext two days ^re those elements 
we had already initiated in fiscal 1978, and only the largest of 
'those elements. But in the few minutes that I have I do not want to " 
talk about those projects, because I think the experts who will be on 
the panels will be much* better prepared<to do that. I would rather 
review ^ few of the broader issues. 

Perhaps the most critical* issue is why, after ifoughl,y 15 years 
of evaluatJLon, demonstration and research activity related to youth) 
we Have a Bill in which Congress says we do vtet knowiwhat works a,nd 
what doesn't, and that we have *to experiment in order to provide^ the 
answers. Given this past record r why do we /think th'at anything will ' 
be different .15 years from now? Why do we think that we will ^ome 
up with^the answers this time around? Without being critical of 
pas^Tef forts, I thinTc we can^ll agree tha^ there' were some shortfalls-- 
almost all of which* were unadvo^Ldable^-which might be corrected.' 

f^irst, because there were. -fewer institu*tionG that delivered 
employment and training prograir.s in the^early 1960 ' sT^xnost of our 
efforts in the past depade had to dp with testing de:?ivery approaches, 
institutions,' and service* packages . We spent a large amount of 
dollars on demonstiration projects that were not meant to measure 
impacts, but to assess the feasibility of ideas. 

In effect there was no real research commanity in 1960. We 
could not have called a group, like this together and have expected 
the same wealth of experience. There were -just not as many persons 

s out there with the talents that we have in this room. Also in ^he 
1960's, there was very much less emphasis in(\ the administration or 
in the Congress on testing, evaluation, assessment and utilization 
of the results. in policymaking. This Congress in particular seems 

^to be directed toward wanting facts' and figures and information 
as a basis for policy; Th,e administration has that same perspective. 
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contrasted with the Great Society of Lyndon Johnson, where evaluation 
results were often a mechanism for sellin<^ programs rather than 
asses^sing where dollar s^ should be allocated* 

' ' ■ ■ \ " 

We, therefore,, had in the 1960 *s a number of isolated demonstration 
projects focusing on administrative feasibility issues with very little^ 
pre- or post-testing* Even in the few isolated cases where results 
were assessed,, the sample sizes and demonstration conditions were tod 
limited. We tried out 10 or 15 different projects with different ideas 
rather than 10 or 15 projects all testing the same idea in order to ^ 
create a large enough sample size and large enough variability' of 
conditions to find out whether or not a simple notion h.ad promise. 

»r In the 19*60 's, when evaluation ^ and assessment efforts began, I 
think we were naive about the complexities. One reason was the antici- 
pation that the benefits of -social intervention would be sb^large 
tbat they'd overwhelm the technical unpertainties , such as the control 
group selection issues. We believed that there would be a 50 percent 
return on our investments so that technical issues would be largely 
irrelevant. Wtien you are trying to assess a 5 or 10 percent return, 
then the variabilities of those issues becomes cr/iticall^ important . 

In the 1960 's, when we did have control groups, we did very little 
in the way of random assignments to alternative intervention. We, 
always chased after the Holy Grail of assessing overall, rather than 
relatdvc i^ipacts. The relative impacts, which are much easder to 
test than the absolute ones, are probably more important fo2t policy- , 
. making where the alternative of doing nothing is rarely considered. 

It was not until the end of the 1960 *s that we began to realize 
the importaace of the longitudinal aspects of impacts. Programs were 
rarely followed up for 6 months, much less for 12, or 18 months*. -Again, ^ 
it was not realized that longer-term impacts would have to be considered 
. to justify the investments. . . 

The problem v/as that the longitudinal evaluations raised th^ 
ante. It, was so expensive to do a longitudinal evaluation and so 
difficult' to design the programs ^om the start with such a long- ' 
rangi^ focus that this approach was rarely adopted. Except for the 
National Longitudinal Survey and Supported Work, we were rarely 
willing to spend the dollars fa: long-term follow-up and careful 
experimentation. CETA — whatev^"^ its delivery benef its--certainly 
underminded knowledge ^developments . CETA is an excellent progra^nV 
but it's very hard to evaluate what happens in a program in whio|i/: ^ 
you don't. have seib categorical regulations defining sets of act- *^ 
ivities that' are consistent^ across areas. It is difficult to find 
out what happens in « that decategorized box to detern\ine the marginal 
effects of difterent types of interventions. You alpost need categorical 
apprjDach^s of some type if ^ you are going to do large scaJLe demonstrations. 
' If you are not, going to do* large scale demonstrations, th^n you fall 
back on the scattered demonstrations which do not yielH^ large samples 
ar indications of replicability . 
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I think that another problem^of the sixties, one that v;e'll 
always face, is that the Congress and the Administration frequently** 
ignored or perveried the results of evaluations. In the Job Corps, 
whicTi is contained, in OYP, the war stories are legion about the ^ 
"goo^ old days" — how teams would retire into"^ the back room to prove 
that the program worked and have it by the next day's testimony. 
That's not the posture we can get away with any more. > Nor would we 
want to. Then, salesmenship was all-important. Accountabili1;y 
lagged by several years. 

' Othe^ times, 'significant results were simply drt)ppe.d. For 
instance, 'the Department of Labor conducted some admirable stutJies 
on how to wqrk with offenders — some of the best work that has ever 
been done in this area. But we never got those results either into 
CETA, or in what's been done in LEAA-OJJDP or anywhere else. The. 
lessons did not translate* into policy. 

At the end of the sixties, the attention simply turned away 
from youth. There has been some research dope on other subjects 
that is quite good, but the youth area was not programatically * 
popular and therefore the research community followed the dollars 
and public in-terests. ^ • 

. The result was^hat when Congress began to consider youth legis- 
lation in 1976^-when it decided^ that the youth employment problem 
was again a- priority — answers were not available to some really 
fundamental issues. I do not mean refined answers; I mean even 
"ballpark" estimates. For instance, althou'gji the overwhelming 
number of persons 22 and under in CETA are in work experience com- 
ponents, no one can tell whether we're getting 10 cents of productive 
output on 'each dollar of wages and salaries, or 50 cents or 100 cents. 
If a maior benefit of work experience -is social output and we cannot 
provj-d^any answers about the value of output, then it's very hard 
for cCongress 'to make a decision 6n whether work expi'erience programs 
arifi- wise or unwise ^investments. 

-\ 

Jt is also questionable whether service approaches are worthwhile. 
In the Youth Employment and Demonstration Project Act, there are really 
tv;o schools of thought expressed. Under Youth Employment and Training 
Programs, Qvery job we create has. to be "enriched," — it has to^l^e a 
'career employment experience, with transition services , efforts to 
overcome sex s^^ereotyping , 'counseling, placement, occupational exposure 
and the like. Yet, another j)art of YEDPA, Youth Community Con'servation 
^nd Improvement Projects, -says ^u can offer no transition servicejs^ 
that the aim is to have kids vror'king hard, sweating, huil'dinga tangible 
products. Thus, in Congress, therte were two views about what worked 
and what didn't work. Services do not offer a product other than 
their impact on^YvOuth, \7herea3^n work experience the youth - who work 
provide some product to society in terms of their output. In choosing 
between work and service approaches, you have .got to know how much 
the output is worth arid hoy much the services impact upon future 
employability. We don't know answers to either of those questions. . 
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Another^question is yhether, or the degree to which, resources 
should be targetted. YEDPA was* not a targeted, law. This Administration 
has gone far beyond the legislation through the regulations, wfe di, 
that consciously because that's t^ie policy that this Administrati 
believes in, yet that belief is founded on equity, not on efficiency. 
The law was very unclear. Some of it said -.you should • s'erve the persons 
who are from families with 8 5 percent or less o,f the^ lower living 
standard. Other parts of it said services or work ^should be available 4* 
to youth whatever ''their family income. Other programs were to the 
poot only. The targeting didn't reflect just a ;^ack of good legis- 
lation, it reflected different views about what the mix of enrollees 
should be. in fact, the Department of Labor was directed to spend up 
to 10 percent of Resources under Youth Employment and Training Programs 
on experiments to demonstrate whether disadvantaged youth benefited 
by being in programs v;ith non-disadvantaged youth 1 So there are those 
who believe you should target and those that believe you shoulcj mix 
enrollments. We don't know the janswer to which is the most ef fec tive of 
those strategies. ^ > , ^ 

Finally, we don't know whether youth should be in school or out 
of school. Should you put your money in prevention before dropping 
out, should you put it in alternative education systems outsttte 
traditional sch(^ls, or should you put your money in non-education 
treatments for dropouts. All three approaches are mandated in the 
legislation. There was no answer about the best interventibn point. 

There was a basic question about how many ^outh really wanted 
to work. Interpretations ranged across the map based, on the same 
statistics'. This is a critically important question. If youth 
don't want wprk, then what do the unemployment numbers'^ mean? If^ 
you offer jobs, how many will "take them? How rrtuch are personal 
impediments, the ll^ck of desire or commitment to blame and how mijsh a^e 
exte^ftal factors such as the simple lack of jobs? 

"We also haven't any answers about the sut>stitution^ issue. This 
is one that we're struggling with in public-service employment. We 
have made some inroads, but— there still remains a great deal of un- 
certainty about what added dollars buy. in terms of add^ services. i 

How much do you add to employment when you create jojbs? .Do the | 
workers displace pthers? If we are judging; our ef fiectiveness, then 
these net efforts must be determi/ied. We have absolutely no answer. 

Another fundamental kjuestion is whether an intervention cai^ be 
effective that lasts more than six or nine months which s^ems.to b^L - ^ 
the longest of any of our interventions. In th^Job Corps, which ^'^^^ 
can offer youth up to two years of ehrollmei^,^ the average length or \ 
stay is 'arfound six months, is there any way "^to develop^a structured 
series of^^anterventions of longer duration that produce a quantum leap^ 
in employajDility? Can we go against the grain of labor markets and 
against the grain of the developrqental process as well? ^ 

These are really fun(3amental q\iestions and I do not think we 
have the answers — or even some* best guesses-^'-yet . I J^^^ mentioned 
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some of the reasons we do not have the answers and why we were charged 
by Congress to learn more.- But what are we going to do specifically 
to answer these questions? This is w^at we'll discuss in detail over 
the next two days. At this junction, I only want to point out s.ome 
of the significant differenced 2:elative to what was done in the 1960's. 

, First and foremost, the delivery and research institutions are ^ 
much more sophisticated and much more developed. We can put a' massive 
program like Entitlemeni: in place_ with a careful research design almost 
overnight because CETA prime sponsors ar'e' so sophisticated' and because 
there are research intermediaries. Entitlement v;as a massive impl^~ 
men-tation effort. In some cities, over 6,000 youth v/ere employed over 
a several month period. 'That's a massive phase-up — tripling the number 
of youth ^ployed in the programs in these particular areas, i It could 
be done because .the delivery and research instil^tions a^lready existed. 

Another important difference relative to the 1960 's is that we 
have a Congress and an Administration that's ready to carry out struc- 
tured investigations . Congress has mandated an analytical posture 
and supports the attempt. As a result of , this, mandate , it has be^'en 
possible to develop and implement a structured conceptual approach. 
It inay not be the best conceptual approach, but at least we tried to 
set out a plan at the outset to coordinapte disparate^f forts, so lhat 
at the end of the^road they could be drawn back to^thfer. As we move ^ 
to the second year of YEDPA, we are trying to tighten that plan, to 
integrate- it even better, to fill in holes and to make it work. Because 
*OYP has control over most youth programs in a sing*le location, it is 
able to link Job Corps and SPEDY as well as YEDPA efforts in order to . 
look at youth programs as a whele. 

1 think that each element of this plan is much more structured 
at the front -end than the average project in the 19q0^. , I am 
talking now in terms of demonstration programs that aire multi-million 
dollar initiatives. The scale has^increased. Where before we would 
do something and we'd end up with a hundred enroll eesr-SO controls and 
50 e)?perim^ntals — f rom which we would try to draw some conclusions, 
now w^'re talking in terms of 500 or 5,000. That increaee in enrollees 
gives an enormous advantage and power- to the present analysis over the^. 
analysis done in the 1960's. " ^^^y^ 

1 think there's ^Iso a much greater willingness to invest longi- 
tudinally than we have ever had in the past. We're investing vepy 
substantial resources in the Continuous Longitudinal Manpov/er Survey 
and in a new National Longitudinal Survey Pane] which we'^11 hear more 
about later. W^'re putting an enormous aniounl of .money in the Entitlement 
program with the' hop^'of doing a longitudinal tracking of the participants 
The research inves^ent is larger to t.^ose in the l^ast. ^ 

Another difference is to look 'much more at alterna^tive service 
approa^ches, that is, to assign , youth randomly 't^o diff^ent yervice 
modalities and "then look at the ^outcomes. This does^not rely oh 
absolute impact measures and does not necessarily weed to determine 
wheth^er a program is a good '♦'^i^p vestment-; rather it/tells whether one 
intervention is more cost-ef ffective than anothei 



I think- we have also been given the authority and^resources to 
cover all bases; One example we'll talk about, today is the school-- to- 
work transition deiwonstration . Here what you have are entry and exit 
tests, follow-up^ measures, control groups aTt-each site, and multiple 
deliverers with multiple sites. You also have impact evaluations that x 
are based on very large data samples frbm all the .projects as well as 
knowledge development efforts at each project, process evaluations, 
and impact evaluations- We can compare deliverer?, different ' . 
sub-approaches, effectiveness within different client groups, impacts 
in different economic circumstances, as well as estimated overall 
effectiveness. All bases are 'covered even if sevetal'Ho not pan out. 

' Will all' this work? VJill we gain anything . from it? We are already 
se.eing.^^f laws in the designs. We are already seeing opportunitie^s for 
improvements. A lot of other questions come up daily fhat-^e woulfi 
like to have addressed. We have already seen a lot of shoddy twork — 
research designs that break down in practice, as they always seem to 
do. I think that in our discussions we* must factor in these realities' 
that out of every ten experiments you try, you may actually succeed 
in following through on the experim|fj^al design in five or three^ or 
one. I don't kiiot; what a good battif^ average is, but it hasn^'t been 
hig^i in the past and we have to expect' that and realize that we're 
going to have a significant slippage rate. 

Yet, we have also seen an enormous amount of highly technical, ^ 
highiy- sophisticated work. A number of deliverers have shown 
unexpected sophistication and a' number of approaches have yielded creative 
structures. feiven that background, what we hope to get v/ithin these 
next two dfeys is a genuine opportunity for us to tafke stock at what's 
going on. ^^There are thirty demonstrations in place, already involving 
, somb 300 local projects.^ They have heeri described a^d*_structured in, 
the' Kn8KW-edge Development Plan'. Somehow, the results have to be 
distilled. ^herd are ^ormous opportunities to'improye designs and, 
to better., coordinate. If we're doing a set of pre-measure.s and post- 
measures in ohe place,- it would be useful to have the same sets of , * ' * 
questions >on the National Longitudinal Survey"or the CLMS • We could 
compare thW results of entryexit with the ov rail evaluafe^ns that 
have sampt* sizes so -much larger and ^Iso have a longitudinal focus. 

> ' , r^— ^ » — 

VJhere like pi^jects axe' being mounted in different areas, we want 
/to have consistency of res.ear(5;h design. That's a very technical aspect , * 
but it's something that have to get to. What we'need today is to 
have groups that are doing like things start talking to each other,- 
starting a dialogue which will hopefully lead to integration oh a 
continuing basis % 

^ * We have a chance to adjust researc4i design^. When' we talk about 
them today, when we. ask questions, these things are not set in concrete. 
There can be adjustments over time. 

I tfiink we ^Iso have a chance to see how the pieces ^fit together, 
that is, to see the overall conceptual framework. The only way we 
are going to be able to "pull the. results together in the future is by 
sitting her^ and' talking about'j^what everybody else is doing. . By doing ^ 

> -4' • 



this we can get a sense of the interrelatibnships" in' a^ll this activity. 

Perhaps the most critical"* issu^' is how we'rfe goii^g to translate 
all the information into knowledge and policy — how. we 'ire going to 
impact on/that and wh^ . This is *'somethirjg we must start thinking 
abQut because policy cannot wait* The questions are going to be asked 
and we have tp provide whatever answers we can when they are needed, 

I think another purpose for the conf erence^and .one that's important 
for n>€ personally is to recognize the work that everybody has done in 
mounting the progr^m^. We are^talking about obligating $2JL5 million • 
in structured demonstration, research and evaluation acti\?ities — 
what must be the largest scale effort of this in [history. We 

are talking about massive amounts of work by very few people. 
Particularly we're talking about Joe ^ Seller and his staff in the 
^Office of Community Youth Employment Programs. , ^ 

I remember about a year and a half ago, at 2^^-00 one morning, 
when Jo^^and I were writing this plan for demonstration research and 
/evaluation activities. He came running into my my ^office with a 
great idea: ^"Let's call -it a knowledge development plan." So, we. 
labelled it knowledge development. Now Joe is affectionately known 
as "M^. Knowledge Development." I ^expect a?n^i hope we will find room 
for improving the work to date, ,but I think you will ag-ree^ that never 
^has^so mu9h been done by so- few. . ' ^ . ^ ' 
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SESS ION I ; YOUTH INCENTIVE ENTITLEMENT PILOT PROJECTS (YIEPP) 

PRESENTATIONS t ' ' • 

MR. EVANS; Q^r first panel is perhaps the most integrated panel that 
you' 11 hear because it deals with essentially a single program — Youth 
Incentive Entitleinent Pilot Projects. * 

Without further comments here, let me introduce Judy Gueron of 
the MDRC, who in charge Of research for^YIEPP^ 

MS. GUERON; The key program feature to remember in thinking ^bout 
the E ntitlement research design — the feature which distinguishes 
EntiTilement from all other employment and training legislation — is 
i:he job guarantee. Any .youth that meets the eligibility criteria 
'for Entitlement is guaranteed a part-time job during the school 
year and a full-time job during the summer. The specific eligibility 
criteria make Entitlement somewhat of an unusual job guarantee, 
different, for exainple, than what's being discussed under the welfare 
reform demonstrations. Under Entitlement the guarantee is conditioned 
not only on income, but on age, residency, schoo!Kperf ormance _ 
and attenda^ce. Specifically, Entitlement is a guaranteed job for ^ 
16 to. 19 year-olds who^eside in the cpmrnunities wh^^re 'the dernonstra- 
tion is taking place, who live in famiTies with income either below 
poverty line or on welfare, youths who do not ^ave -high school diplomas 
and who. agree to r-etuxn to or continue in high schopl or a program 
leading to equivalency , and who meet minj.mum academic and <perf ormance 
requirements. ^ 

The demonstration is taking place .in only a limited^ numbe^r of 
sites. There are seven Tier I programs. These are the larger programs 
- with anticipated enrollment from 3 ,500 to 10,000. They^ in\s4ucje the 
entire cities of Cincinnati and-Denver, and 18 co,unties of Mississippi. 
These are large areas./ Then, thfere are ten smaiaier Tiei?i II pA)grams 
where variations of E/itit^^ment are b'eing explored. 

^At the local level, CETA prime sponsors hdtvd operational 
^ irai^ponsibility for Entitlement, although some of^the prime sponsors 
,have subcontracted direct operational management to othei:^gencies , 
in the locality. / - 

At the national 'leve*/ MDRC is responsible for plannirxg , develop- 
, ment of guidelines,' research, and Offering continuing guidance to 
the prime sponsors in implementing the Entitlement concept. 

* *the demonstration be^an enrollmejr»t last ' February . It was slow 
infthe early months. In^April there were about 15,,000 youth enroJ-led, 
2?>P00 in June,, and, at. the latest count, 33, 000. About 30, 000 of- 
*t?fi6'se were in the seven 'Tier I sites. ' 

^ In authorizing Entitlement, Congress listed a number of specific 
question^ that they wanted addressed. These are listed i^i detail in 
the papers •The. issues can be grouped conveniently into those concei?n- 
ing program implementation impact , and cost. 

• The purpose of the implementation analysis is to understand and 
' explain the programmatic development of Entitlement sites and to draw ' 
some lessons about "the operational and administrative feasibility of 
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thi^ approachr — What does it^ake to translate a legislative job 
guarantee into an operation^^ program? What's the ability of the 
CETA prime sponsors to manage suchi^'job guarantee? What i|p the 
importance of the different institutional arrangements, such as 
those, between prime sponsors and school systems, and how is the 
private sector , responding to the wage subsidy provision in Entitlement 
which permits full reimbursement of wages in ^thex^iring' of youth? 

' The impact analysis is really directed at testing whether the 
program succeeds in its short and long-run objectivp^. The major 
short-run objjective ia to get dropou^^s to returnyxo school and to 
K§3P potrential dropout's in school. In the longer: run, it is hoped 
that the increased educati(^n ^and the work expedience provided by 
Entitlement will lead to improved labor market experience/ Another 
part ^ of the impact analysis addresses the question 6f participation: 
how many yovith will- want jobs if they are a'vailable/ ■ 

Finally, the cost analysis ^is directed at pro\^iQ'ing total cost 
information- which, when combined w i4h whajt w.e attain on the partici- , 
pation rate measures, will yield some e^timates^^f what it would 
cost to extend E ntitlement nationwi-de, / ^ ^ 1^ , 

This research effort is being ^carried out by a number of organ- 
izations under MDRC*s overall respons'ibility • Researchers from ABT' 
Assaciat^es, with Ernie Stromsdorfer as principal investigator and 
Robert Jarrett as the project mana^/r, are responsible for the 
impact analysis. Researchers^f rOm.JiDRC unde'r Joe Ball and Bill 
Diaz have been working- on an ^mpl<^entafelon analysis and the cost 
analysis is being designed, 

MR, JARRETT : The impact analysis is focused on three px^incipal 
'areas: The first is an examination of participation^ in the proqrams*. 
The second focus is on short-term effects on youth educational 
attainment, labor fofce status and earnings. The third is /on the 
Idnger term impact on youth — the postrprogram education, labor ^orce 
status, and earnings. , ^ 




There are three principal sources of data: First, there is 
a program information system set up by MDRC to j::ollect routine data 
on program treatments, *on hours of work, wages, earnings, types of 
jobs, etc. Second, there ane d^ta from school records, concerning 
attendance, tardiness, performance, and types of school programs 
being pursued. Third, ^nd most important, there is a longitudinal 
survey of program eligibles being conducted,. at eight sites, four|^ 
of the seven large Tier I sites (Cincinnati, Baltimore, Denver and 
rural Mississippi) and comparison sites not participating in the 
Entitlement, This survey sample cpj^ers all eligib'^c youths, not, 
merely program participants, means that we- not o.ily hav^ a 
treatment 'group of p^rogram^pa^rticipants , at the pilot sites and 
a comparison group at cerfiparison sites, but we have a third group 
of non-participant>-<tn the pilot sites, '^^his design clearly allows 
us to mor3...jiir^'f*€ly get at outreach questions dealing 'with part- 
iciiia^&^ also gives us much stronger techniques for controlling 
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potential response and site biases. The survey will yield a rich 
data base for assessment of Entitlement, but for other research as 
well. It includes approximately 8,000 youths and their parents. In 
addition to the baseline, three subsequent waves are planned in 19-79 
1980, and' 1981 so there will be a three year longitudinal data base. ^ 

The data set is fairly extensive, including socio-demographic 
characteristics, detailed work; schb61 and training program 
histories, information on attitudes toward work, education and 
themselves, incon\e informatioi^ both on the youths and their 
families, work histpries of parents and a limited set of fafnily ^ 
interaction data. ^ ' ^ * ^ 

To return to the principal research*^uestions , the participation 
survey allows us to, address twd specific questions posed by Congress 
in YEDPA, regarding leve;Ls of enrollment . and the number of youths 
provided employment in relation to those who could have been 
provided employement under the program. In one respect, this is a ' 
very straightforward analysis. V7e will have information on the size 
of the" total el^,gi±ile popluations in our study sites and information - 
on the proportion' of ^he§e who participate in the program. It's a ^ 
relatively -simple matter to deal with participation rates by different 
characteristics of youths, different backgrounds of youths and to 
talk about the participant population in contrast with ^the jion- 
participants. But it is much more complex because participation 
is not simply a binary .variable-- joining .or not joining. It's a 
continuous variable and some youths are going to .enter the program 
and drop out .quickly, while' others may participa|:e intensively through 
, high school. The extent of participation matters greatly. It will 
affect impact and cost estimates. An ^applied participation model 
must be developed that deals with participation as a continuous 
variable. Anot|ier complexity is that we're not looking at one 
program going on during the school y^ar. During the 'summer, youths 
get fullrtime employment and they're normally not in school. During 
the school year, they are in school and tl^ey're getting part-tir^e 
employment. The. treatment is different. The range of opportunities 
and options facing the youths as they make decisions about what 
they're going •J.o do, are different. Thus it* is necessary to keep^ - 
track of the difference between summer periods and in-school ^ 
periods in the analysis. Certainly in the participation* research 
we .would expect participation rates to differ for summer and in-schqol 
periods . " . ^ ^ 

We're going to be doing a preliminary participation study, 
reviev/iSg participatioft' during the startup phase of the program, • 
during the last spring and summer. ^A major report will be produced 
after the second wave of survey data has been completed and retrei'ved. 
This survey is now scheduled for the spring of 1979. We can't 4o 
a final repTort/ really, until afte^ the third wave of survey da.ta is 
^received. That report would probably be available at the beginning 
of 1981 and we'll have youths eligible' to join the program up to 
close to that point. Our second wave of survey data will not capture 
the last six ta nine months of that potential dynamism' of entering 
^ anc^ leaving the program. ^ 

N ■ * 
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The short-term 'impact analysis was originally perceived as 
two distinct studies, one on ip-school . impacts and the other on 
employment impacts in the short-run* It' quickly became evident 
us that these- really are two principal dimensions of one larger 
consideratitpn — the overall time allocation of eligib].e' youth. - 
Essentially ' youths have a'finite amount^ of time *they can spend on 
various activities and ij they ' re xonf rented with* an Entitlement 
.program which offers a job, they have to make- some choices about ^ 
how they're going to use their time "and abqut what's important f 
to them in their time* They have to divide between work, scftooi,^ . 
school -rei?rted study, extra curricular and leisure 4:ime ,activites . 
One would expect that there would-be a considerable n-uihber pf tradeoffs 
made, with working, youth cutting back on studies or, leisure or 
time in school. We'are very interested in this dynamic" and the 
behavioral r^ponses-of youth in budg^ing t,heir time in response to- 
the program. We are developing essentially^ time allocation model 
which will "generate the estimates of intermediate program impacts. 

\rhe scheduled reports on m-schgol impacts essentially 
paraipelsi the participation reports with a, major report at the begin- 
ning of 1980 and a final report^ coming at the begj,nning of ^1981 
wh^n we've had a chance to observe the program running its full course. 

Finally, we will have a post-program labor supply and 
employment mo(3,el ^nd a model' to estimate the demand foj secondary 
education. In^fthese estijfnations, essentially the intermediate or 
short- term*NDutpn:ts from the previous study become final program 
inputs in the post-program models. Our' thinking has also been ^ 
undergoing some evolution with regard to post-program impacts. 
Previotis'iy^ we tended to think post-program impacts as post-secondary 
impacts with the .focu^ on graduation from high school, continuation 
of post-secondary^ education and post-secondary eii^rience in *he 
labor market, as principal impact dependent vari^Bles. We have^come 
to r-ealize, that we may have a population of youth in this program 
who are- in the program for a loTlj^.. period of time, who leave the 
program,' but who are ^till in hicjh sphool or in some fprm a 
secondary school. This can oqcur because the program is oot 
only* school-conditioned, it's also age-conditioned. You leave the 
program ^hen^ you graduate from high school, you also Sheave it when 
you turn^2(l years old, which shouldn't seem^ like much of a problem, 
but, when ypu consider that the program' s primary target groups ^ 

are dropouts or potential dropouts who. are frequently older Jthan 
their grade peers, it is likely that they- n^'ay become ineligible 
for the program while still in good standjing in school and yet not 
haying graduated. We don't know the extent that this VilT ^ . 
-occur. If it is a substantial group, we will have to broaden our 
thinking of pQStt^rogram impacts, vt0 include essentially in-school 
impact measures as long-term impact measures. 

The postvprogram ^aferiMy will be based on follov;-UD v/avefcritiie 
third and fourth wave^of longitudinal survey. The study will, be 
completed shortly after the completion of v;ave ^our and will thus 



be available very early in 1982. 




• There ane other- possible effects of^a c^nging program. For 
example, v/hen Entitlement was getting started, there was considerable 
discussion about changing the eligibility criteria. A number of pro- 
gram operators wanted the , income standard raised from the poverty 
threshold to 79% percent of the BLS Lower Living Standard. In some 
cases, this was because they were simply having initial _ difficulties 
enrolling youths; in many others it was because the Entitlement 
poverty critera were different from those used under CETA. If the 
changes had been made, they would have seriously jeopardived the 
i/esearch because the youth samples had been drawn, and most of them 
l.ad'been interviewed. With the change, the sample would have been 
ro longer representative of the eligible population for .the • . 

program .and all aspects of the research would have been jeopardized. 
Research considerations were. a primary reason for not responding 
fto the request "that, the income criteria be changed. In the future, 
/however, other operational issues and pressures may arise which 
/ affect research" designs. Research may not always win out. 

MR. DIAZ: The implementation analysis is interested in under- 
standing and explaining the programmatic development o^ the Entitlement 
demonstration at the sites in order to draw lessons concerning 
operational feasibility, both generally and under varying 
site circumstances. We are approaching this by first documenting what 
occurs and then tryir^g to explain these results in J^erms of the 
capacities and the interest of t.he operating agencies, their 
accust;omed ways 'of doing business, and their prior and current 
operational relationships. We are also examining the effects of 
local political and social factors at the sites. 

The implementation research begins with program description- 
program content and operations, how many youths were enrolled, what 
are the characteristics of enrollees, what kind of work they do, 
which "agencies ar^ involved an'd so on. Next, we'll be looking at 
operational factor^ that we think^ill have some effect on the 
decision of youth to participate or not participate in the program^ 
These programmatic factors will obviously supplement the impact ▼ 
analysis. Third, we're going to be looking at school-primel 
"^""^onsor linkages under Entitlement; documenting , analyzing the roles ^ 
of these two primary! agents in the Entitlement of f er>x<^hool 
systems and prime sponsors have generally established a number of. 
working relationships over the years and Entitlement will provide 
these two institutions with the opportunity and _ 
necessity 'to coordinate work experience and school services to^ 
greater extent than has usually been the case with in-school work 
programs, we v;antto see how this relationship operates and why. 
Fourth, we are going to,*be examining the adapablility of the CETA 
system to implementing a large-scale job guarantee concurrent 
with ongoing and possibly competing program responsibilities. 
Fifth, we will be looking at the Tier II innovations. Most of the . 
'research is concentrated on Ti^r I. where the majority of the money v 
is being spent and youth will b6 -participating. However, we are 
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going to carry out a systematic documentation of the program ^innovations^ 
such as entitling 19-to 2 5-'year olds* and working with unwed 
mothers. ^ ' . 

These five areas are being covered by the general implementation 
research. They will be supplemented by special implementation studies 
in three areas. The first is private sector work experience. We 
are going to be doing a special study examining- the role of the 
y the pr ivai^ sector in the demonstration. Under Entitlement prime 

sponsors, c^ and have provided up to 100 peiTcent wage subsidy to 
the private sector to provide work slots. We want to examine these 
arrangements to determine their potential for assessing ,^obs in X 
the private sector. Our second special study will be examining / 
Entitlement in rural areas. One of thp Tier I sites is in rural 
Mississippi. Among the Tier II sites, Steuben County, New York, 
Monterey California and' Alachua County, Florida, have large rural 
areas within^ them,. We want t^ examine such things as job-creation, a 
potential factor affecting youth, and other issues relating to 
Entitlement in rural areas. Finally, we are also goi^nq to be looking* 
--^at the quality of work under Entitlement, MDRC will survey a 
sample of work sites to develop a profile of the quality of the' 
work experience of the different programs. The importance of the \ 
entitlement work sites lies in their ability to provide an 
exemplary work experiei>ce that develops and reinforces the pos^'itive 
^'X aspects of work for youth. Therefore, we tentatively indentified 
fiv^ factors on which a sarmple of at least. 5 0 at each 
Tier I site will be rated. These factors have been derived and adopted 
from the literature on job quality. They are, (1) the content 
of the work prpvided; (2) 'the organization and management of the work 
site, (3) the level and nature of supervision (4) the youth per- 
ception of the value of the assignment, and (5) the value of the 
work to the S'ponsoring agency or firm. ' ^ 

T^e implementation analysis must address f uodamental questions 
about the meaning of the Entitlement concept in operational terms. For 
instance, the conditioned job guarantee necessitates tight 
monitoring of eligibility. An open-ended Entitlement where an^yone 
eligible can theoretically demand a job* combined with projections 
of participation that are bound to be off because of the upredict- 
ability of youth response to a new program, means that both 
funding ageiK:ies and prime sponsors have to be concerned about 
keeping the Entitlement str^tly limited to eligibles.' This 
requires the creation of new systems for monitoring eligibility. It 
mdans s-tringen^ enforcement of requirements for proof of eligibility 
in the form of supporting documents. Inevitably, such ^ - - . 

monitoring and .proof requirements may also act as a discouragement ^ 
to participation for inaivixJuals^ Parents, for example, for a sense 
of privacy, may be unwilling^ to reveal income information, or they 
may find income statements too complicated. ^ 

Another operational issue results from the school-cbjiditione^ 
' ^ aspect of the job guarantee. There is a need to establishl school j 
performance and attendance standards.. It means little if students 
^ return or stay in school ' simply in a pro forma" way without' 
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progress toward a degree as intended by the YIEPP. Early efforts 
reveal that school attendance and grade standards gove'rning 
school participation for the general school population generally 
on't exist. This is due to the general reluctance of schools to 
uspend or .expel students except for some overt act, such as 
hitting a teacher or committing some other crime. Therefore/ 
most prime sponsors have had to negotiate with schools for the creation 
of grade and attendance standards specifically for Entitlement eligibles. 
Not only was this process time consuming but the standards are 
frequently vague or ignored. ' . # 

Thirdly, Entitlement as a job guarantee implies an operational 
• ability to 'quickly provide the guarantiee, a job to those who come 
forward. A large-scale time limited work program would also carry 
some urgency to its iit^lementation. This seems particularly acute 
in Entitlement, however, since the program has visibility. For the 
most part, pri^me sponsors have attempted to recruit eligibles as 
ra'pidly as possible to reach protected levels of participatipn , 
and with somesgxceptions high enrollments were quickly reached 
* once the program got underway , but especially at the Tier I sites, 
there were delays in processing from enrollment to placement. 
This happened usually because the large numbers enrolled were 
too much £orS±ie processing system. Speed seems to be given prio*iiity 
over preparedness. Our analysis must separate such start-up 
problems from those likely to affect continuing programs, as well 
as the long term effects of too rapid a phase-up in some sites. 

Finally, there -is a question of how the job guarantee 
is perceived by its potential clients. There are incentives at 
work in the Entitlement program against advertising the program 
as a guaranteed job. One was a fear of being overwhelmed by a sudden 
demand, from youth's, for jobs. At sites where less than J 
complete jurisdiction was entitled, there was confcern about the. 
reactions of non-entitled economically disadvantage areas making 
' ■ their 'dismay and frustration known. Generally, therefore, primes 
did not stress a right to a job- in,, spreading the word. 

MR. BALL: First hand observation tends to separate rhetoric 
from reality. The idea of 'an Entitlement as spelled out in the 
legislation and in the various kind of application procedures is 
somewhat different than the reality to date for prime sponsors 
which has been primarily ' getting kids and a lot of slots in 
• a big hurry. What did get advertised was that this is a big ^ob' _^ 

pr-ogr^m and if you're poor, you can get .a job. Prime sponsors 
' went about business as usual in some cases. Experience with the always 
rapid buildup of SPED^ and with very rapid phase-up of the . 
PSE program in 1977 was us^ and the primes generally resorted to 
old routines to get the program started. With these mechanisms 
they tended to be able to do it reasonably well in .terms of getting 
enought slots for the kids. It was -helped along by the requirement 
in the competition that final applications submitted to the 
^ Department in December of 1977 had to have a project sheet for every 
work sponsor specifying how many kids, what kind of work, 
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what kind of supervisory arrangements and so forth, so there h^d 
to be paper commitments to cover every projected enrollee. 

In general, the program implementation was not impeded 
by a shortage of slots. The result, however, was a heavy .reliance 
on public and non-profit agency sponsors, most topically those 
which had run summer or in-school programs. Potential for privates- 
sector involvement was not fully e^loited. In-school programs, 
co-op prjograms that the schools had run, of course, invoi^ved private 
industry on a small scal^ for years and years, but this was the 
first big opportunity for a" major private sector participation v;ith 
youth j^ork experience. The prime sponsors diglri't have close working 
relationships with the, private sector. Therefore, they typically went 
.to intermediaries like Chambers of Commerces, the National Alliance 
of B^&iness, and in some cases organizations that had been created 
around problems of youth unemployment in the past, such as, in 
Cincinnati, the Citizen Committee for Youth, in the big' Tier I 
programs, Denver was the one thdt made the biggest emphasis on the 
private sector buildup. By the end of June, they didn't have 50 
percent, but they had almost 30* percent of their kids in the private 
sector . \, ^ • 

Some of the Smaller Tier II programs, which had the luxury of ■ 
being small also pushed for a private sector role. In^Philadelpl^ia , 
7 0 percent of the kids at the end of June were working in the private 

sector and a Igirge percent of thetn were working in manufacturing, 
which is a little bit exceptional for the djemonstration as a whole 
where there's a lot more small retail rfirm involvement . ^ Hillsborough 
County, New Hampshire, wltich is a^healthy economy, with* light 
industrial base, decided to take advantage of the demand situation 
there to try to create jobs above the minimum wage with some skill 
development potential above entry level. The jobs that have been 

^ created there have been primarily in the private sector and several 
of the kids are working aboye the minimum wage. The prime sponsors 
that turned to the private sector > without exception, offered the 
full 100 percent wage subsidy. Prime sponsors argued that the 
business community hadn't been involved with CETA and particularly* 
kids and that unless the offer wef^ sweetened to 100 percent, 
they would not had been able to g^t tsnough private firms to 
participate. In the process of negotiating grants with the Department, 
the primes were 'required to star^t, thinking about and develop plans, 
for the reduction of, that subsijdyT level over time to see if they 

-couldn't begin to mdve toward C^l^t sharing arrangement v^ith private 
firms^ that would be somewhat more in.lirfe with the 50^ percent OJT 
rule of thumb.' The prime sppnsQrs-, haveyoeen rather reluctant to 
change^ the deal in midst/re^m during ^tne period of original 
commitment and we will see whether new arrangements can be negotiated 
when grants-are extended. 
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One price'ithat may have been paid for the rapid start up is the 
limited involvement of unions. The legislation authorizes and _ 
encourages primes to work with unions, to develop apprer^ticeship 
positions for kids. They haven't done it. In general, prime 
sponsors regarded this as a secondary and future mi^^on. This is not 
to say, however, that the prime sponsors entirely avoided the unions. 
The legislation says that if kids are put into jobs which may 
be a potential displacement or, might undercut prevailing wages, than 
the positions had to be cl^red with local organized labor. The' 
same thing occurred whereMobs were restructured. Further- 
cooperation of union%i»wa9' one criteria in the grant applications. • 
Strong efforts were made-^to clear with unions. At those sites where 
the unions, especially public unions were still restive over 
problems created by the 1977 PSE build up, formal arrangements were 
made whereby they reviewed every job description and participated 
in restructuring. In Cincinnati a representative from the union 
local monitors work sites continually, writes up his own 
monitoring reports, and indicat,es whether or not they are 'in ^ny 
way infringing on union interest". So , the uriions have been 
involved in a kind of "damage avoidance" way. ^ 

.-To date, then, the program has not developed a lot of options. 
The kids are, paid the minimum wage. There are 22 kids out of 30,000 
at this point who are being 'paid above the minimum. There will be 
some efforts made in some of the programs to try to induce the 
private firms'to-pay more, but it's basically a minimum wage 
program. In addition, there's very little skill-specific, formal 
classroom training. It's a pretty barebone work experience 
program. • 

Yet it is early and we have- just f'inished looking at a program^ 
in its start up phase. This has been very tough in the Tier I areas^ 
where the massive workload forced a lot of program quality 
Questions -to the. back burner.. Attempts to get the schools more 
actively involved in tying their efforts with the work side of the 
situtation, were not pressed. In part, that was a function of the 
' design of the funding structure to begin. with in which th§ money came 
througfa* prime sponsors and it was not much coming through.' education- 
channels. Mostly, it wa& be^r^use of limited tim^. 

The Department has recognized the need and the desirability of 
" encouraging the schools to do more in enriching the treatment _ and _ 
there proba.bly will be gome money available 'to develop some pilot 
efforts to expand education. t 

• To date however, the schools h^ve not been too active. 
They were asked to cooperate ^jLn certifying, kids being in school 
and reporting on attendance. They have been asked to be flexible 
if possible in scheduling so that participants could work after 
school. Although last spring they were not very flexible, there is 
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some hope that this fall, with a little more lead tirne, they, will ' 
. be a lilBtie more adaptive. So far* observers 'are not very encouraged 
\that the\ schools are going to make a big effort to bend their ways- 
-'to adapt I to this program. ^ 

oWrall then, the program has focused on creating jobs to 
guajrap^e employment. The measure o*f the performance will come 
in this jffear as prime sponsors have more titne, to involve the 
private sector and unions, to work with schools, and to seek 
enrichments and linkages. 

DISCUSSION : . / 

MR. STRQMSDORFER; The 'most distinguishing characteristic of 
YIEPP and it's analyses is that you pan observe the entire process 
of the program operation and impact. There is sequence of events 
that begins when you start setting monies down in^ areas and 'there 
is'a logical structure of actions that f qJ.1ow. -^Thefe is j^., 
a parallel structure of analysis. The key policy questions-^that 
Congress asks have the same time sequence, I've never seen this type 
of correlation written so well into the la.w and implemented in an 
operating program. Each new step J.s related to behavior that went ^ 
before and answers a successively more difficult policy question. 
I would also like to poiiit out that given this close correlation 
between the sequence of events, the policy questions as they develop 
as well as the analysis, that the data that are generated are eminently 
practical. ^ \ 

MS. HIGGINS; Oyer the nexj: year the Administration is 
going to undertake a complete policy review concerning youth employ- 
4jjent with ^ focus on what "we 're learning from these YEDPA 
activities.^ The aim* is to get answers both for budgetary " 
purposes and for future' legislation about what kind of programs 
make sense. ^ . * ' ' ^ 

The Entitlement program involves most of the issues that need 
to be addressed.. On the marcc' level , we first need to know hov; 
many people are in the universe pf need. The Bureau of Labor 
Statistics, 0MB, the Department of La^or and others may argue over 
the same set of numbers but there is a chance to find out who's 
out there, how many there are, 'and how many want jobs. That's 
critically important. 

The' other dimension of importance is the , essential bondage 
betweeji work and education. Everybody talks about the importance of 
linking the employement and hiring and education, sys^tems/ but ^ 
nobody has really determined whether' it's feasible and whether It 
' ma^kes any diff-erence in outcomes for youth. So, I think we'll 
find out some of the answers jander Entitlement. The feasiblity issues 
are cruciaV Can the prime spSnsors and the schools deliver this 
kind of program? Can. the* jobs be'created and&will they be 
meaningful? Will they be linked to education? 

* * , ' ' 

Other questions will, also be addressed with crucial implicatiorts • 
How can eligibility be determined? 'How can alternate education systems 
be structured? In brief. Entitlement is a critically important 
experiment v/hich.will have widespread policy implications. 
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.MR, LERMAN > I know coming fr<^m the Welfare Refo^-m Task 
Force, where we're thinking about some of the same questions that 
entitlements do raise special issues. It-*s good to see that some of 
the*'questiorfs will be raised, for us when we go through our welfare 
reform pilot projects. Some of them will even^e answered. I'd also 
likff to emphasize the targeting implications. If you look at the 
unemployment rates for disadvantaged youth^and you look at the gap 
between their employment and that of advantaged youth, the gap is 
greater for in-school than out--of -school youth. I recall some 
numbers o^ the following ord^r — black disadvantaged students had 
an cmployment/p^ulation ratio of .about six percent as compared to 
about 36-38 percent for advantaged students .- So , Entitlement 
is addressing perhaps that group in society wi^th the highest j^ob 
barriers and a problem which is critical. 

The second general point I'd. like to make is that the Congressional 
mandate is highly specific, relative to. the mandate for other kinds 
of studies an,d other kinds of programs. The result is that gt lot of 
the research de-sign has to be directed just at the Congressional 
mandated issues and not- at what we might set out here and think are 
important issues. 

At the same time, it seems to me that some other important 
issues are left aside and let me just note a couple of them, 
because they haven't received too much emphasis. One is the eligibility 
conditions for such an entitlement. Lt's one thing to have a 
separate demonstration program, but in going to a nationwide 
entitlement, broader equity issues are raised. When you have a pure 
income^est at a certain point in time for entry into a job 
' entitlement program, it tends to create the kind of notch that those 
of us who have looked at welfare programs have noticed and decried, 
that i's, the notion that a family which has an income a few dollars 

above the standard are fully out of the program, -while those just __ 

below are* fully in the program. When you consider going to ^ 
national level, an issue like this becomes very important. 

Another issue is how you best structure a •wage subsidy. 
It's the kind of issue that you'd like to know more about. N<:5w,-with 
respect to the three areas — impact, cost, and implementation--let 
me begin with^the third one, implementation. I think it's 
important that we study the processes and the barri^^s when 2j^f5^___ 
to phase in a program, but T wouTd^iTce to~se^~ wire sTru^"tur^d 
, questions about what specific aspects of implementation should 
be ass.essed. The notion of delivery capacity is just a little 
too general for^ me.. ^ I'd like a question having to^do with what kinds 
of waiting lists we observe or^ f or how long a. period. J would like 
the dimension of private sector ,involvemei>t to to be studied and 
laid out in detail. To say we should have some private sector involvejnent 
where, you staffed out with a hundred percent subsidy, does not 
. answfef most of * the important questions. It would be surprising if 
you couldn't get people to employ youth with a hundred percent 
subsidy. The question might instead be what happens when you 
have a 50 percent subSid^, or a 75 percept subsidy, as opposed to a 
100 percent subsidy. Also] there ark questions .about the extent of 
private sector involvement beyound just employijient . 
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Moving feo the cost side, there has been little discussion ^^^^^ - 
about the problems of measuring, costs ♦ You know, when you f irsj; i:hink 
about it, you say, "Okay, we're going to see what the costs^re."^ 
It's a lot more tricky than that, although I'm not sure/€rf the "ins 
and outs," I would haye liked to have seen a little^dre discussion 
today on that and whether or hot there's going to be- questions 
like the' cost per site, and the factors dif ferentj^ating low-cost . 
and high-cost sites. Secondly, I would like to see. a reference to 
social costs as opposed ^to budget costs* Here, there seems to be 
a very natural way of drawing the time aJLlocation model into the 
analysis, whereby we can see what w^ ^are diverting youths 'from, 

jA^lX^ tyhe y ^T"^ Hrr^wn -Inf-n t^^pg^ progr^^m.ci ^ nn_!fJi£^,.hniLri^--£if-^ _ — 

participation come from pure^ leisure, unjJroductive or counter- 
productive activities or from school studies and other things th^t 
are unthinkable? 



Moving on to the impact assessment, let me'.jus^b say thaj: one 
of the problems that I haye with the analysis is t^iat there is 
an implicit emphasis on the notion, of a job versus unemployment as 
compared \to a job versus other jobs. Only at the very end was -the 
mention of minimum wages pointed out. These ar^e minimum wage, jobs, 
but the problem goes beyond that because when you think of the wage, 
you think of earnings divided by hours worked. Now, we may have 
a good idea of what the earning are, but we've also. got to knov; 
what the real hours worked are. This concern is being voiced . 
very heavily by a member of the Council .of Economic Advisors staff 
with respect to 'the welfare reform proposals. To See whether these „ 
wages are really minimum wages, we have to compatJ^ these jobs to 
other jobs to check the hours worked and the effort everted. I 
think you also have to ask about relative job content,^ . - . 

Another aspect of the wage and job nature question is that even 
though the eligibility is based on welfare participation or 
poverty, nothing is mentioned about the^welfare marginal -^tax rates 
and whether or not families whbs§ yoiiths are in these progr*a*'ms 
have that income counted for welfare purposes and maybe food 
stamp purposes, thus producing low net gain. This might be a 
reason why we observe differences in participation from one group 
to another. I think that's a very important question to look at. 



" Turning to the participation question, we've got to loo*k at . 
the degree of participation, rather than just looking at the particifxints 
relative to eligibles. We must consider the flows into and out. of the 
program, as well as the equilibriums. What about repeaters?^ Is 
it easy to flow out. and then flow back in: That's .the kind 
of question ^Tiat you want to ask. I've seen some interest'^Lng* numbers 
lately that sefem to" indicate that actual turnover between jobs ^ > 

is close to :^00 Percent a quarter for black teenagers and that « 
even for white teenagers , it's extremely hign. So, if we observe 
very low turnover in these jobs, it may mean that these jobs are 
better than the other kinds of v;ork you would have gotten or that . 
stability is a very, very important characteristic of jobs. We 
ought to ask, is this a way to reduce youth ' unemployment , that is, 
to try to improve the stability^f employment. 
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A second problem in loolcingat participation dealsjwith the question 
of eligibility. When you take a samplfe ^in a site of 'a poverty 
population, yoU have to recognize 'that people are moving in and 
out of ff)OvertV. The number of participants oveiL time cannot ^ 
just be compared with the number eligible at a point in time. The 
flow out may no^ equal the f^low dn in a period when the economy 
is improving, so that^yb'u may over-st^te the number of eligibles, 
simply because you>/e^ taken the survey in one^^year and you're ^ 



extra polatinq^j: 



another _ye„ar , 



^The impact assessment is>^also based on the nation th,at the job 
program -has no impact on the eligible non-participants, that is, by , 
adding a huge number of pari:-time jobs to the supply of those jobs 
and concentrating on low-ineome ybuth, yoxa, don't affect those low- 
income youth that don't go into the 'program. I would expect . « 
just the opposite. * ^ , ^' m - 

I wduld like to see m9re on how many are dra\^m from unemployed 
and from those not in the labor force when we attract people into 
these jobs. This is important for generalizing from available 
aggregate census data.^f we had asked questions similar to those , 
in the census", how many are we dx*awing from people who would 
otherwise have been out ^qf the labor force? What is the nature 
of the attraction of the4e jobs to the people who are normally ^ 
out of the laB^or force? 

I also 'would li^e^to .see somfet^^ng on do we reach appropriate 
targets and what are appro"priate targeis^s' for employment population 
ratios fdr these youths. Where Entitlement exists, are the poor 
youth in a, much better employment; situtatibn than the relatively 
advantaged youth or are thei^^st^H in about the same position or are 
they stiil/in an even lower pog'ition?^ I think that's a .question 
that many people are concerned about when, they're asking what 
would be afi appropri^e target in t§rms of increa.sing the jobs for 
poor youtH. ' ^ . * \ ' * 



Fin alTy^ ^ eMniS^^^ impacts 'of a job - 

> guarantee program. Manypeople argue* t^iat it will be minimal/ I 
believe t/iat in, thos^e market segments where there is' considerable^ 
excess supply, what "happens. when you add to the number of jobs is 
simply an increase iji employment and not much increase in labor 
cost. When you're talking abour^ expanding 'to ^ an En titlfement program 
• for the nation ^s a j^ole, i think this question would be .a very 
important one. 
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MR . BALL : Mr. Lerman raised qoaestions about hours of work ' 
and time allocation. In the manpower ancj^'th^ manpower' research 4 
industries, the numbers seem to ^take on a kind of intrinsic value 
and, in that vein, through the end .of Jyne, kids in Baltimore ^ 
have been paid foi^936,887 hours of work give^ or take a 'few, This 
doesn't spe^k to whether or not they're working, of course. We ' • 
have to deal with -what the prime sponsor reported as\ gay time. ' ' 
We're planning^ however, with the*monitors we have pin^'Tier l^^i^S}^ 
to monitor well over ten percent of work sites with 'an instrument 
which is still in open developmental process ; but wliigh we are 
fielding now. We expect to ^tart 'getting Jto about ^ 7Q^ work/ ' - 
sites a month, where we are going to address tTie questijcjn of the 
qi/ality of the work experience, whether or not the k-ids ar-e 'working 
and whether or not there are apparently too many kids 'tor trljfiL work 
that's entailed. We're not goin g jbo atte rpp t to do a' quantitaSL^^-- — ^ 
TTSisessTnentT^Birt we afe""going^ to monitor these work sites f^irly^^^"^ 

closely. " . * ^ • 

- ^ *■ . 

MR'. FARKAS : On the self -selection issAe, we 're .going t6 tty a 
variety of things to pin down the extent o'/^possible self --selection • 
^ bias. There are relatively standard econometric techniques which * 
will be applied to estimate -bias and make ^corrections • If we had. 
instrumental variables, we would, have used them but it isn't ^ 
clear at the outset which would be the critical factors. We willi>do 
a prohix equation and we will^ try to be cjear about what the omitted 
^variables might be^ as well as" the possible direction and magnitude 
of Ipias. That's about 'as good as anyone can do. 

MR. STROMSDORFER : I thinic we're going to do a little bit better, ^ 
but th^re are inherent problems. What's lacking is knowledge about 
the influence of d^iand side elemerrts on , supply of. labor and ^behavior . 
We can control s^j^what by. getting information on demand .character- 
istics of'-the lalSor market and we fiave the benefit of having ^ 
both a control labor market and an expe'rii^ental labor , market where- 
you have participants and non-participants who are "eligible • _ 

The issue is how well can we model tl^e differential supply and 
demand effects. That's hard to do. We don't have the necessary' 
statistips" afid ^ detail that we would like to have. 

MR. PACKER ; I just want to emphasize the need'^for strucfturing ^ 
t specific questions at the dutset of the reasearch. I hope that 
somebody 'has sat doWn and laid out the hypotheses that are going to""*^ 
be 're jected^ br accep ted^ Me,Jb,elie:v,e„jtJie ^program will jnean that , , 
disadt^antaged youth who meet the eligibility requirements will be 
more likely to graduate from high school because of the jo^ 
-'guarantee than if it were not present. Can you tell me now/ what 
.difference in graduation you expect between the controlled sites and 
the test sites — that'^is wh§t difference would allow you to reject 
the null hypothesis that it doesn't make any differen&e whatsoever. 
I hope somebody ' s . gone through and can tell me what are the ten^ 
most important hypotheses that you are testing and what the - • 

standards are for acceptance or rejection.: It i^^ot -^enough to . 
say that we are going to get a. data base or are going to know lots 
of'things about the program. What are the ten mosr important hypotheses 
that you're going to, put most of your inteULectual Vnd^ financial 
resouroes? For instance, how many youngste*rs ar^' going to participate 
in the target population? That question has to tranjslate into a 
hypothesis. If we're very sophisticate^, we might- be able to 
have a quantitative estimate. If the unemployment ratio between 



advantaged and disadvantaged youth is now one to six, then the 
program will move it up to two to six, with a confidence limit of 
X and so forth. 

• 

We must be sure .when we're spending the public's money, that 
we have a scientifically solid approach and the ability to accept^ or 
reject specific hypotheses which are* recognized as the ones that are 
implicit in what policymakers are doing and, thinking about in suqh 
programs. Be sure of the policy questions. If I find out that 
when you "double that employment population ratio from one to six 
to two to six, or when you improve the graduation record from one in 
four ^o two in four; and you did it because the best kids out of the 
eligible population partictpatGd--that is, self-selection bias. ^ 
-^owevei^-, — I^m— not— sure that the Congress will find that that's 
sufficient reason to not have the program. If you could say, . in 
a sense, that if you had this Entitlement program, you'd eliminate 
the difference between the employnent ""and graduation exp^rien^te 
minorities, disadvantaged youth and the rest of thjg population, 
I don'^t think an^^body would give a damn abouj: self -select ion 
bia's. If at the end of the Entitlement program, you can't tell me 
whether you've rejected that hypothesis that it makes a difference 
in 'those two things that I talked about, I think there will be a sense 
that we've not done the research in the manner that was truly 
intended. 

MS . GUERON : Inifial design of the research was based on a 
number oi hypotheses, including the two you just stated. Will the 
program decrease -the drop-out rate of studentsthat are already 
-in school and will it have any^impact on their future employment^ 
^^fter they graduate from school, as well as their future 
education? We do not want to end up v;ith hypotheses that have been 
set out but never answered. In fact, the legislation passed by 
Congress was itself quite specific abotit the. questions' it wanted 
"answered and the hypotheses it wanted tested. 

V 

MR : TAGGAR T : The thrust of the is§ue is not with sample sizes 
' and confidence levels alone, but in the structuring of hypotheses 
that .say if the impact is of a given scale and dimension the hypoth- 
eses can be rejected or 'accepted. I think tnis 'question of how 
you formulate these hypotheses, as well as of how you follow through 
to test them speaks to the fundamental difference betv^een 
statistically significant and policy significant results. 

MS . GUERON ; What we had not done, I think Mr. Packer is asking 
to have done,' is to formulate it in terms of policy significant 
impdct expectations. If . we find that the- results of a ^iven magnitude 
ate documented, what then is .the policy action that this 
magnitude of impact justifies? We should not? just come afterward 
and ^y: "Well,^yes there was some impact on this;" and then make a 
pol iciK judgement . CCuld we look at those issues ibefprehand and 
decide what results would be large enough to reject or accept 
conclusions?" ^\ 

MR. .PACKER : ;i don't want to /ush that too far. Many of tht^'^s^ucs 
are statistical. When will you be able^ to tell u$ whether the number 
vbf kids who come back to school, this month, is statistically 

different, than one might have anticipated without, the program? Are 
^more kids enrolled in their senior year who might have dropped 
' out at the end of their junior year? Will you know in a" couple 
of months or when? , , i ' 



' MS . GUERON; No, all that we know, until we conduct a second 
survey; is^hat we can gather from an information system that 
tells us what's gone on at program sites:. We will know how many 
dropouts returned and how many in-school kids are re-enrolled, 
but we" don't know the net effects, on thos« out of the program. The 
second survey, which will be conducted in the lat:e spring or early 
fall, will allow a comparison between what's happened in the Entitle- 
ment site and what's happened in the control site. 

MR. PARKER ; Will the results be available by the time legis- 
lation must be drafted next year? January 198 0 is the scheduled 
date for submitting administrative recommendations. When the 
budget goes up a year from this , January, if we're going to ask for 
any money to continue youth programs, the legislation has got to 
accompany the budget. So it makes a difference whether the survey 
is in- the spjring or the^fall. If you think the research is going to 
affect ithe next youth bill, then we've got to be writing^ that 
legislation a year from now to accompany the budget that goes up and 
requests money. So it's not a matter of ind>j.f f erence whether we have 
a spr^j^ ot fall survey. . ^^^^ ■ 

MS. GUERON: We ha^ assumed very recentj-y that we were conducting 
the survey next spring. As it turns ovrt^.-^tlie baseline survey 
was delayed and we haven't y.et gotjt^r'^e baseline survey results. 

MR. .PARKER : Why do I heed a baseline survey? Why can't I 
ask the school system tomorrow? I mean,-*it's October, kids have re- 
enrolled in school. I don't need to know their lifeline. I just want 
to know if kids who are el-igible seem to be enrolled more in the 
test sites than in the control sites . 

MS> GUERON : You can do it tomorrow, if you want to answer the 
question with that type of methodology. ^ ' 

MR> PARKER : The legislation must be ready in January 198 0 and 
I don'^T~think we're ^Ing to be able to tell them we spent X- 
hundreds of millions of dollars, but we have problems in the field, 
and you'^^l have to write the legislation on the^^-basis of the inform- 
ation you had before. 

MR. STROMSDORFER : You've been ^skmg twp questions, though; 
You've been 'asking, "What's the gro'ss effect, ^and what's the net 
effect?" The f ixst^^uestion is easier to answer than the second. 
You don't get net without a control group or until you have the 
second survey results. " ) 

; 

MR. PARKER : I heard Mr. Taggart say we had not frozen every- 
thing yet in the program design questions and I think perhaps 
that's something to be debated. Maybe we could do some other 
things. It might pay to do a few little samples. Right now, I 
mean, I think one can do that for a very small piece of the money 
with a limited .Survey instrument in order to get resuLts for 
January 1980. I' guess the one thing that I would like to see, 
soon after this *conf erence, is the policy hypotheses that would 
be accepted or rejected by next December 1- 
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MR. HAUSMAN : fs it your impression at this point. that the 
program is getting at the targeted groups — the people v/ho are ^ 
difficult both to get back to school and into employment? 

MR> BALL; It's njy sense that early participation in March, 
April, May -and June of 1978^ consisted mostly of i-n-school kids, 
because th^y were the easiest to reach in the phase up which , 
started in mid-semester , What's going to be happening September 
October,, we'll shortly know, » 

MR> LERMAN: Is there really a job guarantee? Are ^ere - 
waiting lines?-^re there shortages of jobs? ^ 

MR> BALL: There weren't shortages of jobs up through the 
end of Jun^, but in the summertime, there may have been. Wi^^^now, 
based on our statistic's, that there have been waiting lists, Tor the 
jobs, but this had more to do in the pha?e-up with implementation 
problems than^ job shortages. Kids Had to wait two to four weeks 
at a lot of the sites. Some of that may have had a discouraging 
effect. 

MR . LERMAN : In most of the sites, were there more jobs than 
there were people interested in them? 

MR. BALL : There were more job commitments that looked like 
they could be operationalized effectively tfian there were kids 
bangingon doors througti June. But, 'there's a political tv/ist to 
this question -ef whether there is a job guarant:ee. The prime sponsors 
where entitlement covered t:heir whole political jurisdiction didn't 
have much trouble putting the word out broadly. We've got some 
indication of how m^ny newspaper ads, how much ^time on television 
what the total outreach efforts were and the like. The cities, like 
Boston, and Baltimore and DetJroit, where only a portion of the 
towns were ejitit^d had greater difficulties; Baltimore right 
aWay had problaJjp with city councilmen from one of , the white 
ethnic ne i g hbcjBrood s nofe^in the entitlement area. TPi^ 
held copncil Iwaring in outrage over the fact that^ popr k'a;<fs from 
their neighborhood were not getting jobs. This constr^in^ Baltimore 
■"fromXputting out the word on, a broad basis, because they were 
afrai^ .of the repercussions. In Boston there was the sacii? 'problem. . . 
in betweenXa Roxbury-Dorchester combination of geographic areas, 
given the p^t problems in Boston. So whether or not , there could be 
a job.guaran^fee was a function, of the politics more than the |l 
scarcity of jobs. \ 

/ 

MR; PRESSMAN : I'm not sure whether there was even within the 
Entitlement area, a ^job guarantee for kids who had already dropped 
out. Were they offered jobs? -Did they know about it? 

MR, BALL : They were only entitled ii? thef technical sense 
during the early parts of impl.ementation of .the program. That was 
because we had the in-school eligible population to reach and 
build up as quickly as poss^ible. Naturally there was a tendency 
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to focus^^ji on this group. Over the next couple of years,' and 
even this^^mester , we'll see whether or not the out-of -school 
kids come back to^^errter t?i'e program. 

MS. aUERON : "Less than ten percent of the* kids enrolled in the 
demonstration now are dropouts and the rest' are kids who are in school 
That does not speak to what these ratios will be down the line. 
They probably will change because we know that the initial efforts 
were concentrated on in-school youth and that the effort to 
outreach and to develop alternative educational opportunities 
for dropouts will.be much greater than t%y were lasj: spring,. 

MR^. TO IKKA : This is a 'very important demonstrai^-on 
bccausc~'it"Tepresents a commitment to a serious research component 
of c7 demonstration program. Speaking as- a person who did a fair . 
amount of work on the research desi(jn, it is important to note 
that it was terribly complicated and there were a lot of compromises 
necessit^ated by financial considerations. The statisticaM precision 
of .many of the results that^we'll be getting are very, very sen- ' 
sitive'to the- ;:esearch and design of the total sample size.* 
Answers often depend on timing/' of research output. For-^instan^e, 
if you are trying to detern4pe the impact on/ graduation then you 
will have to slice the sample fairly thin to get a reliable 
statistical sample and have to wait a long time. I'd like some • 
comment on how tl^ose compromises were resolved, particularly with 
respect to the priorities among hypotheses. 

M S, GUERON : The questions which had to be address^^were 
those~in tlfF'^i^gislation . An initial decision w&s made flat we , 
simply can't afford to survey poor and non-poor youth in each of 
the communities and that we^ have to get at impacts on ineligibles 
by other less r-efined means. What we really had to strive for^ v;as, 
'tm^dequate sample of eligible poor youth in Entitlement and 
c6n^rol communities and we couldn't extend it beyond that. V7e had to 
assure that for in-program and transition effects we would have 
a large Enough sample to tackle those questions. V7e have a great 
deal of confidence about answering return to school and school 
retention questions, less confidence about ultimate impact issues. 
One of the things that we feel has helped is the extension of the 
program itself. We will be leaking at extending the research desion 
also. <^ V ^ . - ^ , ^ ^ 

MR. TAGGART: The question of whether an Entitlement is real 
or re^lized^'is a difficult one. For instance, we already^' know .that 
the alter^i'^tive'education systems within Entitlement sites were 
not set up in time to attract ^significant number of youth to^date.. 
We alrea'dy know there's not enough money in the budget for 
alternati;ve education, to tA^ extent that that's an attraction that 
pulis^^Jrrrout-of -school youth and^akes Entitlements a viable option. 
We want to test whether or not if 'we add alternative education 
options, we will increase that rate, of return tc school. So, in this 
coming year, with extra funclinq, we will change the program to 
add educational resources. There will be high and low-impact modc-l s 
to see v/hether the composition of the school system and the . , 

alternative education system makes a difference. Tn other words, 
we will adjust the program mid-stream to c|)rrect known problems and 
to test new variables: * Effects on early enrollees will* therefore 

p. - 27 - 




be different than those on later enrollees, 

' We know that the uptake ratq on ^Entitlement* such as food 
stamps, changes with time. We may reach 3 0 percent of the eligibles 
in the first year^NTj) percent at the end of the second year^and 
a 100 percent at the end of the third. Is there any way to guess* ^ 
the trend in uptake rate? If we assess the eaarly results with what^ 
accuracy can we project theSe into the future? It may be that we 
don't want to provide an-y answer until a year and a half, because 
we do not have faith in the steady state potential even though we 
had accurate statistical measures of impact %o date. Perhaps 
it would be better not to rush the results, 

MS , * GU^ON ; When the demonstration was first initiated, it was 
only .foF"l8 months — a furetty short time to test a new concept. As 
the demonstration i^s extended all of the components may be 
extended over time, so that for example , in determining how may kids 
drop out of school, we can be losing at the sophomores in 
schoal during the first ^year of the program and the sophomores 
in school in the second ypar of the program and subsequently. I 
would imagine that the ^^^program impacts will change, >^ 

The extent tO'which the start-up period will differ in results ^ 
from the later periodVA don't think we can have a handle on until 
we've seen the ^tual data on the program, 

MR,-TAGGART: But is there any way to guess ahead from trend 
or to give ballpark estimates that somehow discount the early results 
We know that \ early resuXts of the first round of entraps are not 
reflective of *what we expect. Can we already say what we expect an^ . 
then discount tl^ results or project them up oj: change them or 
alter them or adjust them in some way in a presentation in .order to 
provide a basis for more reasonable policymaking? Will it be 
damaging if we give out information early, which may not be 
reflective of the reality? Many of our demonstrations in the past 
have been ruined by premature judgments about^ the 
results:/ For instance, this clearly happened in the case of the 
income maintenance experiments. It may be better to wait until the 
right time when there is some sense of t^he necessary disunity on 
the results, ^ * . ^ . 

' * • MR, VSTROMSDORFKR; It is not a good idea to let out results 
prematurely, since numbers are never neutral^-they 're alway political, 
My bias as an academic" and a scientist is to get those results 
out tothe people as fast as possible Just that always has to be 
conditioned >y the reality that numbers are not objective. They're 
always subjective and we've got to be prepared to justify with as 
much objectivity' as we can the nutnBeXs we develop. 



SESSION II: YOUTH COMMUNITY CONSERVATION AND IMPROVEMENT ' 
^ ' "PROJECTS (YCCIP) : DEMONSTRATION AND RESERACH 

INTRODpCTIO>Jt " ^ ♦ 

MR. LEVIN: One of the obvious dangers is trying to reinvent 
the^heel. Instead, I want to suggest that some decent wheels ex- 
ist now. We have <to find out where these are and how .they operate 
and then try to replicate them. , 

Thesecond issue is that of inter-agency cooperation or the 
lack of it, at the loc^al level. And when we get into the issue 
of inter-agency conflict and -cooperation On all these levels, we 
want to ask the question of how much do these strategies resemble 
a strategy of feeding the sparrows to feed the horses. 

A third issue is the' operation of in^ter-agericy programs. 
PRESENTATIONS; 

,MR. KELLEY; The VICI project is small, compared- to what 
we've lieard about the Entitlement project. We'll have an active 
number of youth participants in the area iSi 480 at any given time. 
Throughout the length of the project, we will probably serve about 
1,200 to 1,300 youngsters.. Although it may perhaps be small in 
numbers, however, it's my hope, that we do have a lot* to gain in 
terms of the knowledge development potential, in three areas-- 
replication, evaluation and impact. One of the first things that 
we did was to have a literature search done, concerning replication 
There has been very little documented about the process. The 
anthropologists have a lot of great stories to tell us. ^ For in- 
stance, there is the famous chainsaw story about the primitive 
tribe which used axes to fell trees while the older gentlemen of 
the tribe used to sharpen those axes every night. A very com- 
mitted Peace Corys worker got the idea to t?ring^ a chainsaw to 
increase produc^on. The result was they cut a lot more wood, 
but a whole sect«)r of ±he^r society, namely the elderly, became 
dysfunctional and there were more problems brought in by the intro 
duction and replication of a model of ssmiething that works in a 
very different .context . So one of the tft^gswe are searching for 
is how tAe replication process can be 'achie^fed without dysfunction 
To do this, we must begin to look at programs as havihg certain 
'essential or key characteristics and other characteristics that 
miqht be im::idental to the' locality. It was up to us to tease out 
'the difference, and we did that after a review of proqram^. 



MR , DELONE ; Both of the programs being^^disclissed on this ^ 
panel are funded under the You'cti Community Conse^rvation and 
Improvement Projects or YCCIP. This part of YEDPA has $115 million. 
It is for 16 to 19 year olds and it is for work • experience and train- 
ing in producing tangible physical benefits to OQ^mmunities . The 
great majority of this money goes out by formula grants, through 
the prime 3ponsor system. ^Some $^ million is - held in reserve 
for discretionary grants. 

- y ^ 

• The Corporation for Public/Private Ventures was asked to 
^o one of the^ discretionary demonstration projects. We were 
to identify an exemplary program or exemplary program Seatiires and 
replicate them in a variety of cities ... both to tc^t the process of 
replication and to look at a delivery . system which differs from the 
normal priir^ sponsor system and from the HUD reliance on community 
development corporations . 

^To decide what was an exemplary project, we drew on a 
rather extensive survey of youth employment programs that the Ford 
Foundation had funded and supported in 1977, which included 
literature searches, field survey and a lot of interviewing. We 
had information on a lot of programs that fit the general YCC^P 
theme. None of those programs, however, were models. Either they 
lacked solid data to document that it was indeed a gpod program 
although people felt it was, or the target population was a 
little bit different from the target population called for in 
YCCIP, or there v;as no way once you looked at it, that 'you could 
really call it exemplary. 

THE VIC I PROJECT ; What we ended up doing, was look at some 
features and aspects of programs that reasonably experienced 
program operators felt made sense. What I think became a central 
concern to us and a central feature of the VICI project, is the 
actors .at the local level who get involved. In a program of this 
sort to make it really effective, you need a wide range of people 
involved in order to mount and implement a program smoothly and 
to provide a training experience for young people that does lead 
them somewhere, and that does have some benefit for them. So the 
question of linkages became an* essential part of the program design 
that we attempted to develop and cull from other programs that we 
had surveyed . , ' 

And what the VICI program attempts to do is in each, of the 
eight sites^, is develop a program that has the following essential 
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features: Oae, it's a conunuifity improvement program aimed at 
producing these tangible physical benefits, sometimes emergency 
home repair, sometimes rehabilitation and sometimes weat^'eriza* 
"tion. Two /Kit has strong involvement of the construction 
trade unions in that town and specif ically ^to have journeymen 
instructors Working' in these programs ^at a super,visor-youth 
ratio of one to six, which was in f,ac1: what the- unions felt strongly* 
themselves was the necessaty ratio to provide ^od quality 
iipervision. Three, it is a must ;\that adequate materials and 
supplies monies will b.e available in each site by getting a contri- 
bution from a community developmerft agency or city government. ^ ^ 
And four, guidelines about youth '^lJ;aibility implicit* in the. leg- 
islation are met. Those we^e some orS^bfiu* main features that we 
attempted to put in place through a process of invited bids from 
15 <:ities, very careful ' screenings' of those proposals lots 
of field visits, a lot of work with -them to try to build them 
up and finally approval by DOL, and award of contracts 
through' the prime sponsors by the Department of Labor. 

MR, KELLEY: Eight sit;.es have ultimately ' been selected and 
were able to meet the-'very stringent and imposed -^uid^lines. , 

On>>^of the keys throughout tt)e project wasthafCPPV could 
actual^ draft the guidelines with DOL, and review them, and make 
siHre 'that they werfe adhered to, and could provide technical assist- 
ance in getting t^he projects off the ground- ^ 

We're also responsible for the research,' we could stress the ' 
research and demonstration facet ,of the project, from the very 
begin ing and that has had some long-term 'benefits . (For instance, 
rn introducing research designs) . 

^e will -be evaluating the replication portion more through 
process or documentary ' evaluation, than in any quantitative way. 
We have subcontracted with two process documenjrsrs who will go 
to each site including the sites .which were unable to meet the 
guidelines, and to pin down at eac-h site, what's happening and what 
makes this -program, work and [what ^could be helpful to tl^e fAlturc 
initiation o^ the VICI pirogra^n in new sitosv 

The model, -that we 're-fusing is basically a cas? study model, ^ 
the process documenters* will be especially attempting to document 
the local contex,t, the , economic jc;onditions of the city,jvfehe 
political structujije,^ job opportunities and the like.^They 
will look at the type of work that goes' on, the type of administra,tive 
structure, and the local cast of characters involved. 



Our process documenters are also' going to look at the role of 
CPPV itself and its interaction with local actors. They will alsa 
ijiterview DOL national, and regional folks who are not on the sites 
but are intimately connected with the operations. 

A second area of research is work valuation. Our youngsters 
are going to be putt4^ together and building and remodeling and 
rehabilitating. There^s going to be a lof of very tangible pro- 
ducts. How, do we tag that with a dollar number? We drew upon the 
very generous help of a lot of ^ur co-contractors. We ^re . 
to develop and refine a work valuation methodology which njight 
eventually be incorporated into a formula-funded community improve- 
ment type of effort. We wanted something that could be done practi 
cally, accurately, ^nd inexpensively. ' 

As part of their day-to-day work tasTcs, there is someone in 
.each of our sites who goes out to the work project and doe3 an 
estimate of bow much it's going' to cost, for the VICI project. We 
are asking these evaluators to .estimate not only how ^^uch it will 
cost in terms of them getting the material and in terms of the 
amount of labor including our journeymen and the participants, but 
how much a private contractor would charge, f.or the same -work. We 
are going to check those estimates by professional estimates. 
We will take this as acceptable evidence that this is the value of 
the output, if not, there may be certain systematic correction 
factors that can be built in. How do you capture salvage costs? 
How exactly do you capture overhead from cpjie city to the next?* 
We're excited by the challenge. 

The experiences of- VICI participants will also be assessed 
rela'tive to a comparable group of young people with similar- back- 
grounds who have participated in other manpower programs. At the 
leag^t sophisticated level, we'll conjpare oiir VICI progi^a'hs with^the 
aggregate of 'other YCCIP programs. We will not be able to makfrany 
very har-d and fast inferential cpnclusions to that. 

TWe next thing that we hope to do is in every site, to 
actually -compare the VICI youngsters with 'those in YCCIP projects 
and the HUD demonstrations that are going on. (They exist in half 
our si.tes) . We will use the existing aggregated data, the exist^ 
ing intake forms* and the ^xisting termination forms for this type 
of Gonparison. We witt^se disaqqreqat6d individual data for 
these if we can get it and compare aqairt the i"mpact at tormina- ^ 
tion of, VICT with * these programs- that resemble VICI in "terms - 
of type of work performed and participant characteristics. ^ . * 



Finally, we do hope to come closer to a half decent research 
design. What we are saying to sites j.s that you get your 60 part- 
icipants, screen them, use your own local screening processes, 
get*them,up and moving. After that,* we hope to build a waiting list. 
In addition to screening the * 60 , get 'a few more for backups for 
early dropouts. We then hope to. build a waiting list which will be 
organized by a lottery system. Folks that are at the front end of 
the waiting list will get into VICI or have the opportunity to 
get into our program at some time. ^ 

^ For those youngsters who don't do well in the lottery, they ^ 
will be,^ystematically referred to other manpower programs so that 
in this way, ^ we hope to end up with two equal groups, comparable 
study groups, both the VICI ahd other program .participants . 

, MR. "SCHECTER; The'HUD/YCCIP is a bit different from most of the 
others discussed. One, it's being operated by another federal agency 
which acts as a prime sponsor, while at- the local level it is being 
operated by community development corporations and otfeer kinds of 
neighborhood development organizations. That is, it marks a^ 
1978 throwback to the late sixties with direct federal funding of 
local non-profit operations. 

A further difference is that ^ith most of those organizations 
youth employmertt programs are not their primary reason for. being. 
They're in the' business of developing their own neighborhoods. 
In the economic sense, or a ^broad commdnity developnent sense, 
and housing rehabilitation sense, they have another agenda which . 
for their purposes is primary. We tha*t that makes a difference in the 
kinds of work and the kinds of attitudes at the worksite between this 
program and more traditional youi^h employment programs. 

There were three principal objectives fo;- the demonstratTion . One 
,was to demons^trate how well community developiflent corporations can 
put together resources available from other sources ^ puch as ♦ ^ 
weatherization labor., .materials and so on. That distinguishes^ it 
probably from some of the prime sponsor formula funding opferatibns. 

The second objective had to do with tVie nature and qUaliJby.of 
the work being done. The notion once again being that we're an 
or'qanization that's primarily jjQterested in something lik^e re- 
building it 's' community. It's going to have that &s a goal which 
will affect J:he nature of ^ the work done and the impact its 

business. ' ^ • . . 

r ' ' ^ . 

The third aspect, of the whole demonstration was to compare the ^ 
way this project impacts on organization, participants ^ and the 
communities themselves, compared with host * formula-funded. YCCIP 
projects in these same localities. ^ . " 

This «is an $8 million proj^t. There are ten sponsors spread 
all over the couii^ry in both large and medium urban areas as well 
as two rural ^eas. There's an average slot level at each site of 
about 90, so there ■ are "about 5j)0 positions .^overall . We ^ 
expect to serve somewhere around 1,800 tS 2,000 participants over 
the course of a year. * » . 



The program began operation February 1st, and now we are 
in negotiations about how long each of these will be extended, 
but it looks like the average length of the program in each local- 
V ity will be somew.here around 20 months. 

MR. FELDMAN; Our evaluation consists of three sets of site 
visits, plus analysis of participant data at five of the ten sites. 
Those five sites were pu;K .together in consultation with HUD and I 
think the idea was to look at the ones which were the most promising, 
Subsequently, it has developed that these projects are not necess7 
X arily the best. 



The project design and methods of analysis were based on 
the three site visits to each of the projects and an analysis of the 
application forms and the termination forms from each of the ten 
cities. If possible, we were going to use the same kind 6f data 
analysis 'from up to four comparison sites in each of the location^. 
And to anticipate the end of the comparison problem it is probably 
not going to be possible to find very comparablelgroups, against' 
which we can compare the HUD-YCCIP. ' ' • \ 

One of the major reasons is that there is a very sub^antial 
difference in the scale of the projects. The HUD projects are 
substantially larger than the comparison projects and therefore 
the potential for the coijununity impact and ^he potential for the ^ 
impact on the organization is very different. 

We are to be suppliled with adequate data for each of the ten 
locations on alL i:he YCCiP's and we can make some bind, of comparison. 
Thbsp comparable groups are not going to be there aW s o if t 4ie 
question is, how" do they compare/ in some ways things are stacked 
in fAvor of the HUD YCCiP's because of the options they had due to 
the larger scale of the effort. 

On the positive side, we have been able to devote extensive 
time at each of the five of the ten site locations. We have been ^ 
able to interview people in terms of the basic objectives' that were^ 
described: 

First, the impact on the community development corporation, 
second,' the impact on th'e neiq[hborhood and community, third, the 
impact on the part#icipants • We are looking at impact on community 
development corpo^ration in terms of the structure of the organization 
and the organizational changes which occurred that can be attributed 
to the receipt of the HUD-YCCIP funds. The grants between half 
alnillion dollars and $800, 000 onf the average are substantial 
for many of the relatively small community development corporations. 
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In addition, some of the community development corporations ^ 
were not ii>M:he business/of youth employment , so it produced 
a substantial change in their objectives and in, the nature of the 
organization/ They had to tool up to dp something that thgy had 
not done before. Some were, in the education business, some had 
other functions,, but the receipt of this meant^that they had to pull 
together teams in r.ehabilitation, in weatherization, in solar energy, 
and they had not always had those experiences before. 

In terms *of impact on the co mmunity, we are interviewing 

community leaders, political leaders, leaders of other community 
agencies and w^ are exploring the possiblity of rapdom digit 
dialing within the actual target communities tcP see what perceptions 
X)f random' samples of people with telephones are in those areas. 

In terms of impact on the participants, in addition to looking 
at the nature of the characteristics of the participants, the 
nature of their employment while in the program, the kind of 
terminations that they h^d, we are also going to do a, limited 
number of interviews witffKparticipants for their perception of 
the program and on what it meant in' their lives. We are looking 
at how they perceive it now they are in the progi;am and what they . ^ 
anticipate in the future. We willl be comparing that, we hope, with 
the sample people and the comparison of ^YCCIP's. The hypothesis 
w.as that this approach would produce an' effort to put together more 
meaningful work 'and it would have an impact on the community. 
It was not supposed to be just straight employmeitf and in the 
best of the cases and indeed most „of the cases that we have 
visited it is not mak^>7ork. As with the CPPV projects,' an effort 
was made in many of t^ places to hire journeymen supervisors, ^ 
in some cases from the unions, a^d to provide a relatively, low ratio 
of at least les^ than one journeyman to ^en participants. 

Let me ^um up in terms of what^~^neral findings h^ve been 
up to this point and I would stress that these are interim findii>gs. 
First, that in the five cities we hav6 looked at, most ofXthe 
projects seem to be working well in the sense that ±hey h^ve- been 
abl^ to tool'-up to do things that man^ of thei^ have not done 
before. We were relatively surprised at how rapidly organizations \ 
v/hich'had no experience implemented the programs. Second, in spite*, 
of the fact that this vas jio^ supposed to be a social support . ' /- 
effect, services w^^re prbvid^d in many of the places. 'It has 
happened because the CDC were prepared with other programs to 
provide social support. • ' ,^ 

The work seems to be going relatively" well and m most cases - 
it's meaningful work.. There seems ?,to^ jDe a trade-off between the ',,.*_ 
impact oji the institution! and the impact on' the commuriity. The^ 
greatest institutional impact of the HUD-graftts occurs in the cases 
where the organization had little experience or small resources. . 



ERIC 



- 35 - 



i3 



Those that have had experience doing this before were able to 
take thi"^ money and plug it into ong^bing progranvs of rehabilitation 
or housing relocations. * 



Among the- most important .imjpac,ts on the , participants seems ^' 
to be the receipt of .money by the participants and the Work 
experience. But in addition many o£ the jobs' demand the kind of 
work performance Comparable to" unsubsidized employment. C^arpentry 
skills, tile setting, plumbing and - the like have provided 
significant skill training* « * ^ \ ^ 

The job supervisor seems particularly influential in working 
with the youth. The journeyman seem to have developed very ^ood 
^eiaL ionships wi L h ^he youth". They seem to^ s^rve economic ~r oTe 
models because of the high income — -'in the area of $30, 000 a year 
and ^n^ ethnic status which is usally the same as for participants^. 

Obviously the programs where the participants are engaged 
in complex work are providing greater opportunities for skill 
acquisition but at this point it seems significant that whete 
participants are engaged* in complex .tasks and whei:e the craftsmen 
participant ratio is not high, the outcome seems fairly positive. 

Although it's still too "early for a definitive comparison with 
formola-f unded YCCIP's, I think we can say in general the HUD- 
YCCIP's are larger than the compariscy groups and that the scale 
of the project makes it easier to have substantial community 
impact '"that for -the prime sponsor YCCIP's. 

DISCUSSION: 



MR.^ StfFtANK; Why do objective research on youth programs? 
Large sums of money .have been allocated for massive quantitative 
evaluation effort yet no one is asking what tKe pitfalls xy£ such 
research might be, or whether it is even appropriate to what we 
are trying to study. 

The object of the research is a network of youth programs, 
The first problem then-^ is that some programs are not^omparable , 
others may not be replicable. The desired outcomes are Bpggy at best 
and ofter tot ally out of the contro-l of "the program operators. 
For instance, in employment training how many . program \graduates 
are placed upon completion? The factor of how jobs are distributed 
in the labor market is beyond the control of the training 

^hprogram. The outcome o|_pl.acements may in fact have nothing 
to do wiOTi what the job^raining was about. The objective social 

•science ^research model jrtay turn out to be more of a burden than a 
beacon for , policymakers . 



Let me see if I can t_ell .you why. I will also argue that the y'^^^^T^ 
most important part of the program experience of participants ' 
may not be at all quantifiable and therefore not subject to traditional 
outcome measures. - . - . 
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There exists a terrible tension between doin<2f objective 
evaluation and trying to make a program succeed. A basic variable 
. required, for a successful program is that the staffv^n its role a"^ 
teacherTtrainer, counsellor or receptionist, be* st^ngly, • ' i 
' •« or better yet evangelically committed to what they are doing. Thai/s 
the elecricity, • the juice if you please, that will supply the- ^ergy 
to make the program light up. That might ;bethe most important _ program 
variable and if it is absent nothing at-arTmay happen. That will 
put researchers in a tough spot. Are they .part of th^ program commit- 
ment or are they just objective observers?' If their objective 
^ -abs^rvairrons i^tcate things are not going as well, as the staff 



believes, then the staff may just lose it's :uice. If the data 
suggests things are going well, this implies that if the program 
is i^fiplicated it Vill do just as well everytime. But _ if what is • 
reallv making the program- a success is the zeal arid commitfnent 
of the staff, then merely reproducing the operational structure can 
by no. means assure success. How do you replicate the juice? 

As I listened to comments on how tKe various youth programs were 
J. developing, I had a feeling that what -was being sought was ^ome 
quick success.- Seniority and tenure pei^mit me to risk the 
> prediction that there will .not be any measureably dramatic 
successes<in.any of these projects At -best the results will 
show' 50% success to 50% failure— 'the hai'f-full, half-empty glass. 
These- kinds, of' figures will not excite^^our sponsors on Capitol 
Hill. Soi# serious- folks on" the HiU m^TBe^happy to discuss such 
results but it does notjpake good,^ prfess-^ cop>'. If it is. / 
/ -instant hula-hoop .su6ce^ that isiwanted afid you can simply declare 

' ■ "as a gesult of €he eitfei'tlement ^ogramr all of the kids who 

participated are b^tcl^'in school: we are 100 percent - successful" , 
the- pro&onen-ts.will love wpu. "'They will then assert that every kid 
who is about to drop out of. schooir.will choose to stay m an?3_all 
thpse who dropped- out .will go bacfeJ.to sdhool if we just plug m 
the entitlement program. ^ 

I am obviously exaggerating" to. makf ^-jJ^nt. Looking for 
quick fixes has little to do with know Xed^ge" and a lot to do with 
" public relations. If you want* to crea.t6'-^a- strong, positive image, ^ 
I would urge you to exhange the research for a public relations 
department. I am not suggesting that P.-R. i|an evil thing,. This 
Gountry runs on public relations not on objective- research; but, 
1 would suggest to evaluators that public relations is not your 
problem— you were hired to develop knowle^g:j|^. That was a phrase 
Joe Seiler thought of and he is to be c/^dit^d for the good idea. 
Substantive learning from a program as, itf develops might produce 
bound information and perhaps, eventually, spme important 
insights, but it will not produce sex^ V^ct^cular results. If 
that's what is sought after, do -public ^rN^ons not research. 
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For some time at the Ford Foundation we have been looking at 
exemplary project---programs that work. In some of the paper that I-, 
read for this meeting the question "does it work?" "appeared quite 
often. That is a tough question. The existential fact of what does 
or doesn't work/ depends on what you meaa by "it works". In the 

-> exemplary program search I was struck by the number of community 
developmCTit programs that do work and quite effectively. MDRC's 
program of supported work is an example of a program that works. The 
data show that while the people are in the program they are doing^ 
very well. The next question-the objective researcher may ask is 

! what will happen when they leave the program. This suggests what 
is needed is a longitudinal study.. Well o£ course we could 

, study thfese people right after they leave the program, a year later, 
five years later, on into old age. That is to make sure the inter- 
vention really "wo^-ked" .But in 'the peroccupation with post-program I 
data we Ynay lose sight of the ljuality of the experience of the 
participants while they were in the program. 

I am not opposed to longitudinal studies yet I am not aware 
of any great insights that have come for example, from tH^ Ohio 
State materials. It may be interesting, and I am sure we have 
learned some things, but it did not give us the E=MC2 of employment, 
and it neven will. The problems of unemplo^^nent are intricately 
woven into too many other social and economic problems to be 
solved by simple mathematical formulas or 'correlations . Longitudinal 
study researchers will agrue "these people did well in the supported 
work program but what about when they leave?" That question 
scares the h^ll out of program operators because^^hey know they have 
no control over that. My response would be, "v;ell, that depends." 
I| depends on the level of unemployment, where they Tive, their 
ra'ce.or color, how much discrimination exists in that labor market, 
and a long list of variables over v)hj)ch employment programs have 
little or no control. 

i" 

Not only a-re longitudinal questions the toughest ones to 
answer, it is also difficult to prove that whatever post-program 
effects we find can be directly attributed, positively or ^ 
^ negatively to the pj^t^gram. This suggest^ to me that in^oSder to 
understand what is "working", we need to know the experience 
of a program in a given time frame, and accept that* as valid 
information on the basis of which to make our judgements. 

Sometimes I hear researchers say, "I have a hunch." I would 
define that as a felt sense or the existential experience of what 
is. happening. Often I find hunches more interesting than some 
conclusions based on longitudinal data. Numbers may be accurate 
but not insigiltful. Ahunchmay giVe us some interesting insights 
in to what is actually occurring that cpnnot be expressed in 
quantitative terms. \ ^ 

Anothe^'r corner we pasint ourselves into comes from the tension 
betveen advocacy and so called "objective" research. I do not agree 
with those who have suggested that^ there was no research in the 
sixties. There was "plenty of research. Often the problem was that 
many of the people who were researching were also advocating the 
programs. There were social sc'entists who believed in ^cial 
change. Oft times when they developed hard data that did nQ,t fit 



their high e5cpectations or what they are advocating, they simply 
made sure that some of the data died in the printouts. 



Mobilization for Youth, a first large scale social 
experiment was a good example of what I desc^ribed. Millions of 
dollars worth of research was never published. My hunch is that 
\^ did not come out nearly as good as expected. It probably 
showed that half the kids benefit from the progam, while the 
other half did not. Perhaps only 30 percent of the participants 
benefit. The funders would not have considered that a success. To 
the planners and researchers who were yearning for the thing to ^ 
work, it was a disappointment. Mobilization for Youth was going to 
be the proof that juvenile delinquency could fie solved by creating 
opportunities •^'when the, data could not demonstrate that the ^pro- 
gram had "solved" the problem it mysteriously faded away and v/as , ^ 
never published. Was it conjecture that the sponsors would disown 
MFY if they had seen the research outcomes? Would they have considered 
it a failure because only a f ew\ kids got jobs? One kid even made, 
it to Harvard Medical School. We had his picture all qvet the place; 
he was our success story. My experience tells me that if a^ 
program can succeed even for only thirty percent of the kias, then 
it has not failed. 

Based on this kind of experience I am concened^ about the 
proposal to do a cost benefit analysis of the youth employment 
rehabilitation projects. Save the taxpayer's money! I will give 
you the outcome of such an analysis. It is cheaper to contract the ^ 
work out. What a cost benefit analysis fai^s to comprehend is ^ ^ ^ 
that not only are old buildings being rehabilitated by kids who • 
never saw a hammer, a nail a piece of plywood^ and could not 
read a ruler are learning how to do construction work. It is an 
education for them, so of course it costs more to rehab with that 
crew. If I want to play the numbers game f coulcf cut the training 
cost for each person and reduce the total cost to only a few 
thousand dollars. The trouble with th^t course of action is that 
while manipulating the numbers, you sacrifice the substance of tehe 
program to make it look good f or U«e funding source. That is 
not to say that program content^^^^^s not researchable , but we do 
'not know how to measure that ^prdntent and it's human value. Too , 
often we substitute quantification which may in the end tell us 
more about number manipulation than about the r^orth of a jj . 
program. ^ 

We can only measure what we know how to measure. This is an 
important principle but it is often overlooked. What happens - 
to the young unemployed or school dropouts in these programs may 
not registe^on your questionnaires. Their own experience may prove 
' to be, the-^inost important source of your data even if it does not 
fit the purvey. For them it miv^ht be a life-saving experience that 
somehow did not register with the researcher. How dO^you gather 
that, kind of data? You start by learning to listen. Don't los^^ 
what happens to people while they ar^ in the program. It sounds^ 
funny to talk "experimentals" and "controls." These are the part- 
icipants in the p^gram. 
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I mentioned the coininunity development corporations (CDC's) 
as an example of programs that work, For almost a decade 4:he 
Ford Foundation has been supporting a number of CDC's. Our 
overall estimate is that they ^re successful • A key factor that we 
were testing with "these CDC's grew out^^of some things we had 
learned during the' years of the anti-poverty programs • We found 
that* to deal with complex social problems through ^hort-term' 
funding was a serious mistake • Operating agencies cannot focus on 
projgrams if their major concern is- where their next grant is 
coming from. The foundation decided^ to do an experiment with Ibng-* » , 
term funding and the effects(|^ve been most positive. It has 
conviced me that social programs heed a jninimum of three years 
of funding if we are to expect any serious results. The notion that 
we can produce inst&nt results in six months or a year in lu-^crous 
in the face of ^pur years of experience that tells us otherwise. 
It is simply uhreal. I appreciate the political pressures from 
"pending legislation" and on the folks who have to" run up the Hill 
to tell the "good news^' of .what happened, but knowledge development 
is the goal, it will take time, and tfee results will probably 
not be spectacular enough to excite anyone on Capital Hill. 

A final comment on quantitative research and social psychological 
problems draws on the experience of one of our 20th century giants 
of learning, Piaget. He did his research on his own children. This 
^ is a remarkable feat when you consider the impact of this man on 
the world of learning. His universe was observing, his own kids ^nd 
how they learned, but he was able to synthesize a lot of 
observations and develop an astonishing insight into how intelligence 
develops. We are in Hesj^erate need of some synthesis of successful 
youth employment program experiences. The numbers may give us 
quideposts but they will not tell us the what and how of programmatic 
success. We can not be satisfied with the numbers alone. Go 
. beyond them ...go to the existential side and see what you jgan^find. 
, Perhaps it will be insight. 

^ There was another diss"ervice that * we anti-poverty warriors 
did ourselved,in thle sixties. Sometimes I think we were our 
jDwn worst enemy. We had not read McLuhan arid we did not know that the 
medium was the message. We insisted that we were social scientists 
not hucksters so we, had' to tell the truth. It turned out that 
the truth as we told.it was misinterpreted. We acknowledged . that 
some things did not work and our poli±ical naivete even said whole 
programs were ill-conceived, believing this would result in 
an objective social science discussion of "the problem." Instead 
program funds wejre stopped and generalizations sprung up like 
. "the War on Poverty Was a big bust." This was taken to mean that the 
social problems we were attempting to solve were intractable. What 
followed was first benign u(^<]l(*rt and now the* politics of Proposition 
13. The creditabilfty gap^ did-i^Qt come only from the right-wing pros^. 
It came from liberal democrats who siad "nothing works". When I 
hear people say "the Job Corps, Neighborhood Youth Corp, Head 
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start, MDTA have all been failures", I am Surious, 

«. • 

What is my evaluation of thes^ programs? I would. say some of the 
Job. Corps camps are good and some are not. Some Neighborhood Youth 
Corps programs are bad, others are good: like most institutions in 
society, some are great, » some mediocre, fair or poor. Some of 
the graduates on follow up studies come out very good, others not as 
.vjgood-, other look poor ^or have dropped out al€ogether. 
Most institutions divide up abdut this'-^way,- and I am sure w^c^^vrilA 
find the same outcomes with the youth employment programs. 
Therefore, look carefully at the data but look beyond it. Be aware 
of the political^ implications that will bf drawn from .the 
research findings, but most importantly look for the positive ^ 
impact of the program on the partic;kpants . 

MR. BARNOW ; The concept ^of the VICI project makes sense. 
Once we have iJi^tified programs that do seem to work then it's 
important to know how'^to replicate them and I think there's going to 
be a„lot of difficulty, in the project in going through the replication 
partr.^ In some senses, repiic^ing, ttiese training ^programs and ^ 
work ^experience programs may be somewhat easier than replicating 
preschool or high* school or grammar school programs .with a particular 
curricula and the like. Vet^ the- liter'ature on educational ^ 
programs suggest that it is quite d^ficult to try and figure out 
wh^'t are the keys ^to replicating, jkhat are the key things that 
you are trying « to replicate? I just hope that it turns out that 
we did indeed isolate 'the key^ factors under VICI. 

An important^ issu6 heire^ for our experience and existential 
philpsophy, is whether or not" failure to replicate means a failure 
of the program. It's quite possible that Some of the features 
of the original program were just not suitable- for elsewhere. It's a' 
serioue question as to whether or not it should be replicated in 
full undeiVall conditions. Alsc>, I hope they will determine' 
whether or, not failure to replicate certain- aspects is a 
positive- or negative factor and I h(5pe that will be locked at. 

The VICI project w^ll- also look at work valuationT^t 
reminds^ me of the man described by Oscar Wilde who knew the price 
of everything is the value of nothing, because what they 're- 
doing , is" determining the prices things would cost in the marketplace, 
, .biit that^s not necessarily the value. I think it's a very difficult 
issue to try to get into the valuation of public sector, common, 
property resources. ) 

. - ^ y r 

THe thi^ issue is the pos^-program analysis and here is one 
area where I am quite concerned. It seems to me that both the 
,VICI project and.the HUD-YCCIP projects are going to look at 
termination data, rather than using any post'-pTDgram interviewing. 
I'ts v^y dangerous to only look at what happends to people at 
termination status. For instance if you look at .the positive ter- 
mination rate, under public service employment , it ' s typically 
around 30, 35 percent. If you look at the rate of employment six 
months later, it's around 60 percent . "Termination success depends 
partly on f)] .jccrnfriU efforts anct so it's dangerous to just/ 
pick the one point in time, tjnd ospecin] ly the immediate date of 
tQrm.ination. 
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I think the question of whether or not we. want to look at the 
poit^program data and how seriously we look at it, depends orr ' ' 
what pur gpals are in the program. It's not clear entirely 
whether we should be concentratij;5:g primarily on in-program benefits 
or post-program benefits, I have the belief that Congress 
is really interested in the post-program benefits and that seems 
to be jfi/hS€^s in the law. ^ 

Another issue that we want' to go into in detail, is whether 

of not the people in various programs are comparable. How do 

people ga^^ into the HUD-YCCIP , versus a plain old ordinary YCCIP? 
Are they similar people? - *" ^ 

0 

In the HUD-CDC project, it's important to try to figure out 
what the goaj-s^^re and set priorities, so we can fxigure oiit what 
we want to look at. In this case, I was most interested in the way 
that HUD put it together rather than DOL\i I g^uess Wiat's just my 
own bias, HUD goals were to look at' the impact of ytne program 
both on institutions and the. community, while 
DOL had an alternative^ goal of comparing it to ofcher delivery 
approaches . Right now a lot of our money goes out to the prime 
sponsors and we don't give ♦them technical assistance and we have 
to learn ways to work better so we can h^lp them. This is true not' 
just in the nev/ programs but it's true of the larger CETA efforts 
in Title I where some prime sponsors feel, that we haven't helped 
them enough in ways .of exploring the programs. 

One concern I have is this comparison of the HUD programs ^ 
with fehfe other YCCIP programs especially because of the fact that 
the HUD programs are bigger. I don't think we should compare HUD 
programs v/ith 500 people and then go say, the big program has 
more value. What'^we want to do is determine .whether th^re are 
economies of scale . . - ^ . 

MR. PACKER: . . ' • ' ' 



Why don't we just take this money and lay it qjat in formulas 
and hire some kids for three bucks an hour* instead of paying for high 
powered research? -The answer jfnust be that those researchers are going 
^^to^ontribute an answer to Congress and the policymakers that will' 
allow them to allocate their resources better. 

My judgement based on my experience is .that Congress is 
dying for some good information. They' live on PR because that's 
all' anybody** gives thiem. Solid research work, even lou^y research 
work has an impact. The ideas stick there, and you'd be amazed 
about how little information Congress will use because it has to 
have some .sort of information to make a judgment. 

Hopefully, we'll get some answers and we v;ilJ>Ndeci(^e that 
the mor\ey we spent on research is worth more tb^iT^roviding a 
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couple more jobs for these kids because ^that's what the alternative 
is* ^. ^ ' , * ' 

4i . * 

MR, FILER; One thing that I think we have got to b^ prepared 
for is that the research can have negative as well as positive 
findings* People had better be prepared' for the fact that you may „ 
prove that your things really are as ineffective as we thougl],t* 
They might be and that's a perfectly valid outcome from the 
research that's been funded* The purpose of the money that we are 
al^ being given is to find out what is working; not how can we 
best present something that looks as if it is v;fc>rking* 

MR* PACKER ; Most of the research v/hich has been done to date 
could"lTever have been justified in science* It's not been good - 
enough* It is not good enough in my judgement to continue to 
finance research at the*.expense of jobs for kids, unless it does 
better than it's done in the past*^ We hav^. had lots^of mrs- 
adventures with re^arch, p.artly because the researchers didn't 
ask questions that""Wtei^»jvere going to answer, and didn't 
follow through. 

MR* KELLEY ; But our schedule of inputs into policy must be 
realistic.* It must take into account what we know aj|put the 
lags of the data coming in* A lot of those data that you want are 
not to be ready on time* 

MR* PACKER ; You could do a study starting now and finish 
it in three or four^ months to know it more kids have come back to 
school in the places where there are' entitlement projects* That 
could be done by just getting our there and going to those schools 
and asking teachers in the ninth grade, whether more kids are 
coming back than usual from the eighth grade. That's why I'm 
here, you must have answers on time. Legislation is coming 

that is going to lay o,ut billions of dollars to be spent in '81 
and '82. My impressioW of the people on the Hill is that they really 
do waRt to spend it i|i the right sort of wa^Kand t^ey deserve to 
know 'What you can find out before they lay oux a couple of ' 
billions bucks. • ' • 

MR. SHRANK ; I think that a lot of research could^ have be.en 
,a lot sharper than it v;as. We will say that next year about this year 
but I would take your example as a perfect example of what not to 
do. I don't know what the attitude of those ninth grade teachers 
are, but; these kids wei-e bringing back, maybe those yare precisely 
the kids the ninth grade teachers really didn't want in 'school, 
and pushed them* out/because they were difficult leatners* So for 
me to go in there and just say to ninth grade teacner's, bow 
many kids came back and so on, may give me the wrong kind of c ^ 
information. So I don't want to do that. That to irfe is not the way ^ 
to solve^ the, problem* You set an example of what rhot to do. 
I also don't know what the ,school 'records would s^ow, but I would be 
very leary to go to Congress and -say the program worked or not on 
that basis. ^ ' ^ 

MR. PACKER ; ^I certainly don't/^want people to think that the 
DOL money is beinq laid out there in a fashion that if the answeijs 
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are not in .accord with our thinking we want to suppress the wrong 
answers, because we dbn't. We really do want knowledge develop- 
ment. It may be presumptious in a sense to put those words on it, ^ 
but I think it's not a bad description. We have tried to develop 
knowledge and I think philosophically, I am a. strong enough believer 
in the ^democratic process to say as analysts we just find out as .^i^ 
much as we can- I believe policy^^ak^rs , the true policymakers, the 
politically elected policymakers, zo who the people i\^e given 
s^o\/ereignty , will do the right thing- The intent of the research 
m*oney ii^^to find out as best we can and to give them the information. 

MR- KELLEY : We are all ^in favor of doing research that 
is relevant, of a high technical quality, as quickly and as cheaply ^ 
as. possible. I wanted to go back through to the question that's ^ 
been-'raised about the value of output work- No one would 
hypothesize that kids will do this work cheaper than a private 
contractor. That's not what we're arguing. We're arguing that 
there is some value to their output and how much is a critical factor 
dn estimating overall benefits and costs. We're not trying to 
answer it in -a real economists' sense of what the value is- We are 
looking at the alternate supply cost and there's a reason for that. 
One good reason is that it's written into legislation that the work 
that is^ done in these programs must be work that would not be done 
by a private contractor. Presumably then there is work for which 
there is no market demand and one reason for that is very often the^ 
work is being done on the homes of vary poor people, who are not 
going to have the money to fix up- their porch or "rehab" their house 
or put on a new roof. / 

If on the other har\d, you argue thatj|here is a» public 
policy goal, that there is a social goal'^to provide that kind of- 
decent living accommodation for citizens in inner--cities, then 
the question is how much would it cost you to meet that" social 
goal otherwise and supply side costs then start to become a very 
relevant estimate in my mind. 

« ' 

MR- DELONE ; We're devoting a lot of energy to following up 
at least 80. participants if we can make the contact three months 
after they finish the VICI program- We want to follow up with 
the same number of non-VICI folks- We're doing a three "month 
f oll'ow-up and longer iff it proves , feasible and resources are 
provided. 
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) m. JONES r We will be discussing five relatively unrelated sets'of " 
projects next.' In this session, I think we should look at cfb'ss- 
fertilization as well a's comparability amoung the data. 

MR. PRESSMAN; f'm from Youthv/ork which is a nonprofit intermediary 
established to assist in the research and demonstration aspects pf 
the exemplary in-school grants' program. One difference between our 
prq ject and the others that will be discussed is the scale and opera- 
tional complexity. We ultimately received over 500 proposals for 
exemplary in-school projects .and had to select between 45 and 50 from 
that. This was the "let 1000 flowers bloom approach." Ten additional 
projects that will be funded noncompetitively will enable us to fill 
in the .knowledge -^velopment gaps that exist in the competitively funded 
projects • 

One of the things we will emphasize in our knowledge development 
throughout the demonstration iS following the perceptions of young 
people over time- V7e also, as an in-school program, are very interested 
in that linkage between the local educational agencies and the prime 
sponsors- The programs that we have helped to fund are supposed to ' 
be exemplary projects, demonstrations, innovations, if you will/ and 
so, in addition to providing ^some 'work, they are supposed do better 
at teaching basic skills, to ^improve counseling and guidance systems 
and to. do a number of other things that affect the overall j^rticipant 
outcomes, rather than simply providing a job. 

We have a three-tiered approach to our assessment. There is a 
management information system that eyery grantee has to implement in 
order to get their money. We also have an intermediary local knov;ledge 
development pi^ce that requires the local prime sponsor to do knowledge 
development^about itself. We provide the technical assistance and 
some of the suggestions about the designs and research questions. 
Finally, we have independent research being done on related subjects. 

I think that we ^ave to recognize from the outset some limitations 
of our research. One is that we're dependent on local knowledge develop 
ment- A second is that we've assumed that there were a lot. exemplary 
projects out there to be studied. One of the first thirigs we have to 
find out is whether, in fact, the selected projects are exemplary., 
v;e, talked a lot about innovations and about the opportunity to study 
innovations and their impact on institutions, but I'm not sure whether 
we'll have all that many innovations to look at. So I think it would 
behoove us, to be very realistic about some of the limitations of our 
research and knowledge development.- 

MR. HEZNICK: I'm from the National Institute of Education. We have 
one of the 17 interagency agreement? that were referred to this morning. 
The Department of Labor has charged us with the responsibility of 
studying the Career Intern Program which is an alternative high school 
program developed by the Opportunities Industrialization Centers of 
America, under the direction of Leon Sullivan. It's 'a program designed 



for 16 to 21-year-old dropouts 'and^potential dropouts. 

DDL is interested in learning whether or not a prog ran\. that has 
proven effect;ive on a small scale can continue to be effective when 
the approach is impleinente& on a larger scale. Our research is intendecj 
to answer four basic questions. What happens to the program itself in 
the process of implementation in additional sites? We're carefully 
studying the implementation process at each pf the. four sites. (1) What 
accounts for the .changes or adaptations and for fidelity to the design? . 
(2> Does CIP <?5nt.inue to .have impact in helping youth when it's imple- 
mented in sites other than the original prototype site? (3) What happens 
to the young people in the CIP that could account for its effectiveness? 
The early evaluation identified what seemed to be the essential elements. 
We had. some hunches. This gives us a chance to find out if those hunches 
are true. It is hoped that the- answer to this question will help the 
Department of Labor and other federal agencies design other programs.. 

Of course, the last question we hope to answer is, "How does this 
approach compare in effectiveness, feasibility, impact, and other 
factors important for policy with other approaches serving youth?" 

MR . TALMADGE : I'm not -sure we are going to answer all those questions 
although wie will certainly address them. Let me start with the first 
one we reefer to.*as the fidelity of replication. I don't think any of 
these projects are replications in the true sense^ of that word. They^ 
are attempts to implement a project that was successful on one site, 
but they are appropriately modified to fit into new situations. I 
think that when we identify deviations between the replication of 
sites and the original model, then we neecj to assess each one of those 
in terms of whether or not it's positive or negative adaptation. 

In^looking at student outcomes, we v/ill have a randomly assigned 
^control group determined -by lottery. In fact, we 'go one step beyond 
that. We, are going to have a matching of students prior to assignment 
to treatment or control groups, so that we will have a matched pairs 
design that will enable us. t6 handle problems axJ.sipig from differential 
attrition. Should, for example, the best students graduate out of th$ ^ 
p^t^pam before we get their post-program scores, anc^^th^ worst students 
in ^jiB treatment group disappear for one reason or another before we 
get their post-program scores, that would leave us clearly in a ^ 
situation where treatment or control groups, though comparable initially, 
would not be comparable at the time we did the evaluation.. 

We have had some probleirls in implementing this" design. There were 
delays in .the program and there 'have been serious 'difficulties in get- 
ting as many kids in as called for in the experimental desian." We IT 
may end up not having any of the nice thingis we're supposed ^to have.^^'K 
We may have a choice of serving some kids or assigning everybody to ^ 
the control group. It won't be quite that bad, but we could end up in 
a situation where it didn't really fnake sense to assign enough kids 
to the control group to give us a meaningful comparison because that 
sj^ould significantly cut down on ourv^bility to. assess the replication 
reffort itself. We don't know exactly'^at we're going to do in this 
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case, but we will resort to the usual tactics of a search for comparison 
groups, perhaps, 'coming from the feeder schools from which the CIP 
interns come, perhaps from other sites that 'have similar problems, but 
do not have a program. We really haven't worked that out yet because 
vife're still hoping that it will be possible to implement the design as 
originally planrijed. 

In the area of trying to relate characteristics of the program to 
specific outcomes, we will be relying heavily on ethnographic types of 
analyses, in-depth case studies of students, interviews with 'teachers, , 
students, '^parents azid community members. We, have ethnographers who 
will essentially live on site, observe as participants in classroom and 
non-classroom situations, and try to tease Out some of these hidden 
relationships for us. . 

In the comparison of costs with ^f^ctivene^, again we will have 
problems. The CIP program is somewhai^^niij^ie^in- '^^hat i is basically 
a high school program — art alternative high school program — that 
provides high school degrees, not GEDs. It's not clear that there is 
another program quite like this to which valid comparisons could be 
madf . However, we will be making what comparisons we can. We will 
be looking primarily at those outcomes associated with participants 
getting additional education or getting meaningful employment. We 
intend to foltow this up for as long a period of time as our contract 
and the initial start-up of the coHorts will allow. 

We are also assessing achievement gains in reading and math. 
V/e're looking at career awareness, self -concept , and sense of control 
over their lives. s ' ^ 

MR. DELONE: The CCPV Private Sector Initiatives Demonstration copld, 
in fact, be called both a demon_stration and an ^experiment. Th^ 
demonstration is an effort .to test out and, in some cases, to^ develop 
'and test out, a variety of programming approaches. We're not setting, 
them up as research experiments, but rather as programs that we're 
trying to make viable and then research very carefully. There's a 
distinction between that and a pure experiment. The test of direct 
incentive subsidies to enployers is more of a sure experiment. 

On the demonstration side, there ^are i^a variety of cross-cutting^ 
questions that we want to try to answer for the complete range of 
approaches.. We want to try to find out whether disadvantaged, out- 
of -school youth can become more competitive, and whether opportunities 
can be permanently increased for them, given a supply of jobs that 
our programs are not going to materially|af f ect . Does'work in^the 
private sector in fact produce better attitudinal adjustment, better - 
skill development, better human capital accumulation, than does 
^ work experience programs in the public sector? 

Central to all that we're doing is the question 'of what the 
private sector will respond to and get involved with. That's a somewhat 
unusual research emphasis, ' but it becomes a particularly critical ques- 
tion with the emergence of Title VII, and the creation of private 
industry councils, which, in many cases are organizations beginning 



ERIC 



- 47 - 



a p^oces^ wondering what the hell they're goring to do when they're 
finished. There is a dearth of good program knowledge and experience 
that's well-d6cuinehted that can be exported and transplanted if it 
works. 

Becairse of our concern with the private sector, we are caught 
in some rather exquisite dilemmas. - One thing the private sector 
doesn't like, for instancy, is 'people from the government coming in 
and collecting a lot of data about what they're doing, which is exactly 
what we need to have ^f or research purposes. We must deal^ delicately 
with that kind pf tension. In all of this, we'll be trying to look 
at firms and what .happens to them in terms of their employment records, 
who they hire as they becdbe involved with our programs, whether t^iey 
change their intake and hiring procedures and their supervisory metlTOds 
and work stryfctures "^to accommodate youth. We will be following youth, 
colle^ihg^^tensive data on them .at entry^ following them through the 
program and, afterward to see if the effects of a particular training 
prograrh or a subsidized work experience show up. 

We'll be looking very carefully at. the process of program develop- 
ment through ^ocess evaluation, trying to find out what really does 
go on. If we do have a program that .shovj^, some »good results, what 
rriade it. a good program? I think that's a key element of thi^ research. 

On the wage subsidy-direct incentives fexperi men t , we aire very ' , 
much at the beginning of what is an extremely coiTtlffitt?ated technical 
planning process and I will not try to^ get ihto the complicated research 
issues that are involved there, except to say that I think we will be 
particularly interested in testing varying subsidy levels to determine 
whether the take-up rate is significantly affected and foiswhich kinds 
of firms. The assumption to the test is that the OJT subsidy, * either 
in amount or duration, is often not a very strong incjpntive for firms 
to hire the youth who are our concern in these programs. 

We are in a very early st^e 'in all phases of this demonstration. 
Among our programming initiatives, several are off the ground, but 
most are st-ill in the planning stage, particularly the direct 
incentives experiment. We [re trying to involve a lot of people out- 
side of our organization to assist us in the technical elements of 
planning this demonstration and the associated research. We still 
have a lot of flexibility to change directions and we, would really 
welcbme input from people who are interested. 

MR. MUCHNICK; The Youth Community Service Project in Syracuse provides 
stipends to 16 to 21-year-old youth who are both out of school and out 
of work, to participate for a year in meaningftil community service 
projects. ACTION conceived of this model of volunteer, community- 
based youth service for three i;easons: First, we thought it was an 
alternative to the' usual way of delivering necessary community services. 
Second, we thought it was a jjiew way of thinking about approaching young 
peoples' problems of the transition from school to v/ork. 7'hir(1, wo 
thought of it as a najor alternative to tho customary ways of 
thinking about organizing national yotfth service. 
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based 

founded upon public values of conununity service and citizenship r» not 
simply the private gains o-f jobs and career training that, the young 
participants ma^derive. Its notipn is that as necess;ary as jobs are 
for young people — clearly they are very necessary — jobs by themselves 
are not sufficient .to overcome the estrangement that young people feel 
from adult members of their community and vice versa. 

So our notion is that the vaX'ue of YCS to the participants is 
surely their personal gains, but also the service that they render to 
their communities — if you will, their acts of citizenship, their 
expressions of civ4.c loyalty. In return, we expect the non-youth members 
of the community to complete the civic bond, to give expressions of 
support and recognition, expressions and commitments of tim^and energy 
that really make the program not just a youth prpgram, but a program 
of total commuhdty involvement. 



One of the ma\or questions, then, was whether or not it is feasible 
for this kind of total commutjity involvement, whether the ideas can be 
learned, Understood, and implemented at th^e local level. For that reason, 
to test that feasibility, ACTION has placed primary responsibility for 
the administration and implementation of the* program on the Youth , 
Community Service Corporation in Syracuse, New York. 

Between the 15th of March and September, we have gone through a 
period of testing the basic systems', a period of making the transition 
from ACTION to local grantee operations, and a period of educating the 
local grantees at all levels — the local board of directors, tFie local 
staff, the local sponsoring organizations of youth projects — in terms ^ 
of wS4>t the philosophy, ideolo^^ and research objectives of the Youth 
.Community Service Project are. 

Youth Community Service initially meant, all things to all people 
and, therefore^ there were never any conflicts at the beginriingV 
It's only when we started to get down to the nitty-gritty of approving . 
a particular project that people began to understand what was differeht 
and what was not different. ' So the vague notions of "do-goodihg, " 
which everyone embraced eight months ago, begin to really wash out. 

We haVe about 150 volunteers in service now. We have another 
165 in the' process of being matched to services and another 200 or 
so who have applied and been accepted for their orientation in October. 
We have developed something on^the order of 400 service opportunities! 
Seyeral hundred more are in the process of development at the moment. 
They cover a variety of needs areas. A major emphasis has been on the 
quality, v/e think, of the projects, and some of them are particularly 
significant. For example, we will . be producing Syracxise's only biling-* 
ual newspaper and, in fact, the only Spanish-language newspaper. 

In^term^ of research issues, I /think there are' basically two 
major dimensions. The first one is the feasibility of this model of 
a voluntary community-based youth service. What is really the capacity 
of a local community to understand and implement this concept of total 
community involvement in support ot its young people? Secondly, what 
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is the capacity of sponsoring organizations,, neighborhood groups, 
public agencies, established .voluntary organizations, and youth 
themselves to develop and supervise meaningful projects that engage 
youth in a. variety of really demajiding tasks? The major thrust is 
^the implementation analysis that ^is being undertaken by the Research. 
Corporation of the City University of New York. That analysis will 
take an' in-depth look at the whole process of impleiaenting and educat- 
ing the Jocal community in terms of wltat YCS is all about and involves. 
There will be ethnographic examinations of 15 to 20 projects for the 
"hunches" about what works and why. These will also consider issues 
of replication — whether or not the project could be done somewhere ' * 
else. The Youth Cojnmunity Service concept involves such conceptual 
changes for program administrators at a local Ifevel that it cannot be 
done on a simple sort 6f revenue-shar ino , no strings attached, basis. 
The question, therefore, ' is what kinds of easy strings could be used 
to get this kind of notion across, but not encumber the program 
seriously. 

The other dimension is the effects of the various program elements 
on the' various types of youth who participate, as well as the effects 
on the community at large. In' terms of impacts, v/e'want to find put 
if youth adopt the philosophy or ideology of community. servicfe . 
Thenv we w^nt tV^ind out more about individual choice in their 
^election of theprojects. We don't match people with 'slots. They 
select the projects they want to serve in,. We will need to examine 
day-to-day activities in the project?, what the youth have, in fact, 
engaged in^ and whether it's meaningful and non-traditional in the 
VteTTfTs we use.' We will look at the type of sponsorship, if it's a 
neighborhood group as opposed to a ci*ty agency; the type and quality 
of the supervision; the possibilities of attracting volunteers f^om 
different income levels, since there are no income eligibility limits 
on the program. . ^ ^ ' _ . ' 

Finally, there is the question of participation ir) service learning 
efforts. Will volunteers who e»|age in service lear^^ing have a better 
experience than those who do not? Will it be\a critical part of the ' ^ 
program? ' , , - _ 

The Urba;i Institute is in the proges*s of developing a series of 
•research designs by which to evaluate some of thesajj^fec^ and exper- 
iences. 0 Because of the proceis of educating and d^J^oping systems 
in the first six months, we djld not proceed headlong into the 
design of research before we really knew what the relevant factors would 
be. We wanted to get some of those systems basical}.y established, so 
that the researgh de.signers would know wha?t, in fact, they should 
evaluate. The major thrust v/ill ba discovery,' not ' specif ically - 
hypothesis testing. There will be some element of comparison with other 
programs. We will be gath(?ring the same descriptive data on our YCS 
participants as. is being gathered in other DOL youth programs. 

MR.. SEILER: The Services Mix Alternative Demonstration Project adcjresses 
the question about the most effective mix of wor'k and services--whether 
youth will benefit more by standing and sweating or sitting dow-n and 
being served. They v/ill be in 3rov?ard County, Florida, Oakland, 
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California, and Los Angeles, California. These are just starting. 

# 

The project is designed to assess the relative effectiveness of 
three basic alternative approaches that prime sponsors frequently use 
to assist youth. The first service approach is the labor intensive 
mode of participation in work experience. » The second service approach 
is called a service support moc^e . There is no work experience. All"^ 
paid hours are in classroom type activities, skill training, educating, 
orientation, group counseling, that sort of t;^ing. The third service 
approach is a mixture — a suppoirted employment mode. Half of the paid 
hours are for work experience; the other half are class-type activity. 

In each of the three projects there are 300 youngsters, 100 in . 
each of the service modes. Each project ha's a random assignment 
process for participants and is matching them in* pairs of three. 
These snatches are established by sex, race and academic proficiency. 
The three pairs are then randomly assigned to each service mode. We're 
also controlling for the amount of dollars taken home by each youth so 
that we don't have the distinction between allowances, and wages. In 
addition, each participant regardless of service m9de* receives job 
deve.lopment and placement services from the sapie , program unit and 
hopefully l^he level of effoift for each will be tlue same. 

The duration is 24 months. The initial two months are for start- 
' up. The last two months^ ^re for the analysis and write-up. In the 
middle, of the 2 0 months, 12 are for program write-up operation and 8 
months are for follow-up. The follow-up includes 30-day, 90^-day, and 
240-day follow-ups. The duration was dictated by the fact that .two 
years was the maximum amount of time that the program could bj^ funded. 

There a^e two major types of information in analyzing ef f ectivene.ss . 
•First, we have a series of beha<^ioral change tests, which are made at 
program entry and exit. These include measures of job knowledge, work - 
attitude, job-seeking skillg, self-esteem, and academic proficiency. 

The second major set of , information is the outcomes on employmeTit, 
earnings, and ipduca^ion in the 3 follov;-up periods. 

V^e Jiave some problems in measuring costs where services are 
received from other sources, but we will try to tie these down as 
much as possible. The behavioral change measures that we use are 
as good as are available, but we capnot get two people to agree on 
how you deal with measuring work attitudes and other things. 

DISC'USSANTS: 

MR. PALM£R: It i*s cl-ear froni the presentations'^.that we're talking u 
about a tremendous variety of projects and objectives. There are ^ 
five categories, but within a couple of those categories,, there's a 
multiplicity of different kinds of demonstrations. I am struck not ' 
just by the variety of activity but also^ by the different assurrptions 
that seem to underlie a lot of the programs. For example, the Career 
Intern Program is really ajL a 1 ternatjve to traditional schooling — an 
attempt to enable peoplp^'^to complete the degree , program. On the other 
hand, 4 ,lot of the Exemplary In-School Programs are an attempt to * 
-2;^edirect the school ^ystem itself in order to serve more saiccessf ul ly 
the youth v/ho.are involved. On the other hand, th^ Communivty Youth Service^ 




Project is taking youth who are out- of school and giving them as much 
as a year's experience, in some cases,, with the expectation that they 
will certainly further their education. Nevertheless, it is entirely 
out of that educational framework. So there are very different assump- 
tions underlying these different approaches as to^what may be needed 
or what our assumptions are about the failures i/^ the system. 

So, to some extent, this variety of activities and assumptions 
reflects both strengths and weaknesses in terms of where we are right 
now with youth employment policy. It's indicative of how little 
we really know. Also, it may be that we're trying to do too much 
under these particular pieces of legislation and demonstration right 
now, and \hat the resources that ^re available for knowledge development 
are being stretched too thi^. ^ There's been a lot of people who've ^ 
mentioned the speed of implementation, the sparsity of resources and 
so on. iStill, it is an extremely exciting and unique activity, l' 
think, with' tremendous promise to be very productive. 

The str'en^ths are the fact that there are a multitude of exemplary 
projects that seem to exist out there now that have considerable promise^ 
and they '.re all being looked at, or at least most of them are .being 
looked at, in one way or another in much more systematic form than , has 

j^reviously been the case. The new projects that are being started 

• seem to me to be indicative of a tremendous ayount of imagination being 
brought to bear on this problem and also a willingness to examine inter- 
ventions that are based upon very different views of what the world is 
'like out there and what might make sense. That is, there's an incredible 
open-mindedness about the whole approach. 




I 'rA struck by the extent of use of 'intermediaries — botl\intermed-* 
iarie^ tnat are nongovernmental as well as other agencies thr5ji^ 
interagency agreements. The intermediaries are really • playing a dual 
role. One is, that they are providing the research capacity to exam!ine ^ 
what's happening out there, but they are also, un many cases, playing . 
a very strong role in terms of getting the operation going. In th^ 
interagency case, there are very different mechanisms, *a whole variety 
of them being used, with more or less participation of federal agencies 
^n'd their counterparts at state and Id'ts^J^evels . I hope that 'a fair 
amount of effort is put on t^he evaluation of these approaches. We 
must try to understand and learn ^tter about the role of these -inter- 
mediary organizations and other federal agencies, in terms of both 
structuring and delivery and the research aspects of the whole effort. 
I can imagine that in many cases, what fnight happen is not simply 
jumping' from a demonstration to a national program, but beginning to 
build on the kinds of structures that are out there now, using inter- 
mediaries a^ technical assistance vehicles affd other kinds of things,- 
to work v^ith program operators at different local levels. 

The capacity, in many cases, simply won't be there, to go from 
wha^t can be successfully done in one area to replicate it on a national, 
level, but over time, using the kind of reservoir of talent and exp^tise 
"that's beginning to grow up outside of the federal government it may 
be possible. A major development in the last 10 or 15 years, is the ^ 
extent to which there ' s. a capacity growing outside of the federal 
goverjiment--some accumulation of the knowledge and some continuity. 
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I hope that this effort really -promotes that even more anh that there's 
some focus made to stand back from that set of inte^Tneddaries and ask 
A * »what role can they play in the future? . What have they done well- in 
this process and what haven't they ^ne well? What gaps are f:here 
for other kinds of intermediaries 7^ perhaps , that are less • research- 
o]?iented and more operationally-oriented, in terms of the kinds of 
assistance that they x:an provide? - ' ' 

Several of the program approaches that are being tried out are 
predicated on the assumption that that particular program may the ^ 
only one in the environment, if it were done on a national level, say, 
or if it were a major program. For example,' the entitlement programs 
are really an attempt to deal with the universe of need among at least 
low-income youth. "The Youth Community Service Program is, in some 
sense, the same thing. Can we create enough slots or service opportunity 
to meet the demand of everybody who would want a job? In reality, of 
course, we're not likely to end up with a worldj^where one of these 
programs would be the only thing that would be available. We're 
probably ^oing to want a whole mix of them. In fact, the notion, 
in some /Cases, is that the youth' population out there is kind of 
monolithip and. either the entitlement program will work or it won't 
Vork for disadvantaged youth or either the CYS will work or won't work. 
* But I think we have to recognize that we're going to need a kind of ^ 
a "different' strokes for different folks" approach. Som^ combination 
of a number of these different^ programs in the environment at the 
same tim^ is very likely to be desirable. 

I don't know how you deal with that ^hen you start evaluating 
these individual programs, but it's important in interpreting the 
results, I *think, to keep in mind that you shouldn't look at them 
as thaugh they're going to be the only program in the enviiranment . 
In some sense, this may mean that they will appear less successful as ^ 
individual programs than the entire group of programs would a^pppear 
or would be in any given environment. 

A goal of the CYS and the entitlement programs is to measure . ^ 
what the demand will hB for these opportunities. In fact, it's being 
talked about even more .abstractly . What is the demand for jobs on 
the part of youth? .Well, I guess one of the implications of what I'm 
saying is that^ these programs are not really going to be able to get 
at that. What^ I think you're g6irtg to be able to measure to some 
extent is the demand in a particular locale at a particular point in 
time for this parti^Qular kind of service opportunity or job. Npw, 
that's a much more limited outcome than we'd like to^ know, but yet 
it's st^ll going to be very helpful for program planning purposes and 
will begin to tell us whether the kinds of results that are coming^ 
.out of the aggregate models using CPS data make any sense at all. I 
" don't think they're -going to eaable us to get precise estimates of / 
what" the universe of need or participation will be, but clocirly it ( 
will be helpful. 

As a third point, I think there's probably going to be an impor- 
tant need for the ability to construct control groups for a lot of 
these programs out of data bases that are totally independent of what's 
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being done in the context of the program. Su'ch constructed control 
groups can bo used for many different programs to enable some 
comparability across programs. I would think that the longitudinal 
survey clearly offers^ this opportunity and I hope some tho.ught will 
be given to how that can ,be structured in such a way as to enable 
different samples to be drawn that can serve as control groups for 
comparison purpose^ to participants in various programs.* 



Fourth, it rs clear in a number of the discussions that it's 
' very. difficult to pay much attention to variations in the demand 
side of the labor market' ana in the economic conditions that are 
going to exist in the individual sites, ^he outcomes you may get 
under tight labor markets in one city or one point in time for a 
given program may be very different from what you get under diffegent^- ^ 
labor market conditions in other areas or at other points in ''the 
business cycle. Rather than expecting .that, in the context of any of 
*thc individual demonstrations, much can be done about this, what's 
needed is some funding of systematic research to try -to expJtore 
^ better--maybe at a more theoretical level — how to get a hanaie on 
'these questions: . "What effects do different variations in labor 
market conditions have on the outcomes of programs like this?" and 
"How can we better measure what we ^me^n by tightness or looseness of 
labor markets for the populations we^e concerned about?" Understanding 
of the demand side and of the broader economic context may then be 
applied to the individual demonstration efforts. ^ 

, > 

I.hav^^ sense that there's an unwarranted degree of expectations 
and optimism regarding the precision and value of what the research 
results that are going to ^ come out of a method like this really can 
be. Therje are a number of reasons for this. It ' s . important to -reee^r. ^ 
nize that, the research' me tl\ods we have' available to us in the ^social 
science disciplines ar^ simply npt very powerful' in terms of trying to 
understand the kinSs of outcomes, the kinds of effects, the kinds of 
interventions, that are being tried*^ out here. ^ 

I 'ye had considerable >^miliarity in the past with the income 
maintenance experiments. There you had as 'simple a^ intervention as 
you can imagine, in terms of what was being tried-^directN cash payments, 
undef very controlled conditions, where you were looking at variations 
in two things: the amounts of payment that was given related to income 
and the marginal tax rate or the rate at which benefits were reduced. 
Even so, after av. set of five or six different experiments under very 
.controlled conditions there was considerable controversy over the ex- 
tent to which we really do understand what the responses of people 
are to > variations in those two different variables. On the basis of 
information obtained, however, the uncertainties have been reduced. , 
We can now say there hasn't been a massive withdrawal from .the lab.QT' 
'market as a result of it and that's an important result. I think 
there will be counterparts to those Tcinds of results coming out of ) 
these experiments. But it's only after 10 years and several experiments 
that you really have the sense that precise estimates of labor supply 
are good enough for use in your cost estimates. It took weUare 
reform that' long and we're at least 10 years behind that kind of effort 
in the youth employment fields, which In most cases, is not going to 
lend itself ^to that kind of precise treatment in any event. 
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Secondly, the speed with w[hich we can get answers has been 
addressed quite a bit and I think it's important to recognize that 
it does take years and years and an accumulation of research and 
understanding • Programs must have a cha;2fce to evolve and settle 
down, in some sense, to get over the early mistakes of initial imple- 
mentation • ^Sb, even where answers can be forthcoming, in ma^^ cases, 
it is going' to take 3, 4, 5, 10 years, and I think Congress n^t be 
told, that^ Frankly, they're going to have to understand that you can't 
simply pass a bill and' expect that 2 years later, you're going to get 
[answers to such critical que3tions as they're asking., in this legislation^ 

Thirdly, a range of different efforts are going on here, but only 
a very few can be talked about as experiments where you structure . 
tljem in advance, systematically varying certain kinds of treatments, 
measu2?ing the results, a^id. then try to understand what their effects 
are* Und^ the best of conditions, when you have experiments, it's 
difficult to get precise research estimates • i^der less cjontrolled , 
conditions, particularly where there's the tension between operational 
concerns and research concerns, it becomes very difficult to get the 
kinds of answers to the questions we're asking and that simply has 
to* be apbreciated. * , . 

A foujth problem is the repl^icability. problem that's been raised. 
I don't' think I have 'to go into that in any detail, except to point 
out it's^ not onl^ a question of replj-cating it because of the uniqueness 
of .a pai;ticular site or the uniqueness of. a particular program, but 
there's 'also a question about the capacity of the system: We're starting 
out in many cases with the '*cream of the crop," fn terms of the prime 
sponsors and in ternis of independent program Qperat^ors, and so forthf 
and trying ^ideas out with them. That doesn't mean, if it works therfe', 
that: you can, in a couple years, just have a national pIK^raF that, will 
work as well* 

As I suggested earlier, the v;ay we get from 'here to there may 
have to be a very slow cumulative process with the kind of expan^on . 
of the network of expertise and bringing up the speed of people at 
local levels, where they do not have that expertise now.- . 

Also, on the funding side, on the intermediary side, and^on the ^ 
technical assistance side, the capacity is very limited. . It's already 
stretched too thin, perhaps, in^* terms of ^^^^LS^t of jSemonstrations , 
let alone what it would mean if you tried to^j^SVg' national Proqrcfms 
in many of these areas within tlia^ext couple of years. So, we have 
to build up 'on that side. I-^^in<)some attention needs tc/^e given 
to that in terms of the interpretation of the results and a *l6t of 
thought must be given to how we get from here to where v/e want to go 
with those programs that do appear to show, the promise oif bfi^ng valu- 
ahS^ on a national scale. / ' ' • * ^ • 

Last, I think there's also a question of our ability to define' 
conceptually/ in terms that can be pperationall-y ^justified , the* 
specific treatments for interventions that we're concerned about. ' 
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Many of them just don't lend themselves to the kind of clear definition 
thcxt enable us to test them out and talk about how changes' in them will 
effect changes in outcome. Jh^ kinds of interventions we're talking . 
abq;^t here are extremely complex in. most cases. Taking "the example 
of '^the income maintenance experiments, there you did have two precise / 
parameters that ypu could control in an operational "^sense and then try^ 
to look at their effects. But "^here, the interventions are really the 
programs as a whole and there's no way to separate out' individual 
components of them in most cases. " 

Some of the discussion seems to suggest that we-' re going to be 
able to vary components or individual elements of these programs ' struc- 
tures and see what difference that makes. I'm dubious that\ that's 
going to be possible in most cases. I just don't ''think that 'the , 
matftodology 6r the resources, that are being brought to bear are going 
to permit that to happen 1 You're just 'not going to get effective 
answers to those questions in the strict research sense. You're going 
to get some important answers out of the more qualitative analyses 
that are being done and from the hilnches that have .been made. It will 
be unrealistic to expect our statistical techniques , a's they're applied 
here, ho be able to answer with certainty or within prescribed bounds 
of certainty, v;hat those variations in particular treatments are. ^ 

dlir best hope is to g^t some overall sense of the effect of these 
interventions at the general impact level, and that is an extremely/ 
important first step^ to take. 

One exception to this may be the wage subsidy experimept, which 
I think is the closest thing to an experiment ihht '^S^s lend itself 
to the kind of treatment not possible in a lot of the other prograiji§. 
I hope that will be given a lot of thought. It seems to be, since 
it's being very slowly developed over time. If the targeted employment 
tax credit passes Congress, as it appears it might, it creates problems 
in v;hat ought to be done about this subsidy. ^ It might be "useful for 
the Labor Department to test that legislation, in the sense of asking 
for some ability to dp 'some experimentation that ties it in with what's 
being done here, so that a year or two from now, we won't just have 
to live with the results of the targeted tax credit as structured in 
the law, but can also say what variations in that particular program 
might yield in differential ^results. 
♦ ^ ^ 

This leads me to the following conclusions. The importance of 
this entire research effort, .at least 'in the short-run, ought to be ' 
focused on first-ord§r questions. We ought to make sure we get answers 
to. suc^ basic questions as: Was iA feasible simply to mount and execute 
the program under* the design condiftiohs that we were tijying to accom- 
plish? V;ho is Being served? Are we reaching the target population? 
Is "it working, in some sense, at tha£ level? That's^ the best inform'^ 
ation you, can hope to get from t^is within the ^next year or year and 
a half. We're not going to get much further than thaj:. I hope that 
enough att'ention is given to. those questions and is not focussed on 
some of the ones in which the\ answers are necessarily going to be ^ 
equivocal. We must not miss getting this. basic information' out of the 



experiments • I'm optimistic that the more sophisticated systems / being 
put in place to provide the data to answer the tougher questions we 
won't be able to answer very well^ will yield the kind of data -we 
need to answer the easier questions. The kind of rigor that's being 
brought to bear ^ because of the quality of research people that have 
been brought into this^ is going to insurd that there will be a 
systematic look at a lot of .these fundamental questions. There are 
'the kinds '^of questions in Which there's a great deal of interest and 
from which a great deal of knowledge can be derived to inform Congress 
and to provide the grist for the PR work that needs to be done. 

The same priorities suggest themselves to me by the schedule 
of timing for the legislation. If I worK<r backwards to inform FY81 
decisions, the administration really has to decide what it wants to 
do by this time next year That really means that we've got about 
six months — maybe nine months — to generate information out of this 
set of activities, to inform that decision-making process, at least 
within the a.dministration . You have a little bit longer to develop 
new information for the Hill. 

But I'd like to reiterate the notion that we shouldn't be pushing 
for more sophisticated answers to the tougher, questions in that time 
period. That would be a mistake in terms of the confidence one can 
liave in tbe answelfs, and more fundamental questions about just what^ 
seems to' be working and what can we do successfully, even though we 
don't yet understand the full impact. 

MR. NICHOLS; We're looking now to find out whether mi^^tream correction 
shouj-d be made in this knowledge development plan and whether the kinds 
of suggestions I might make should be taken as self-criticism as much 
as of aaythiiig that's in that original knowledge development ^plan, since 
I do have some overall responsibility for research design. 

^ want to emphasize the rfeed for consistency across these various 
programs. John Palmer just said rightly that this is a grab bag of 
projects we 're .discussing on thi^ particular panel. There are many 
different projects demonstrat'ing many different things and it's going 
to be very hard ''to draw generalizations from them. 

We want to strive to brinq about *some kind of consistency, so that 
we can not only make comparisons withic^cach of these projects, but so 
we ' 11^ also* be' able^ to make pretty good comparisons of one approach 
against another approach. It should/be our objective to try to compare 
across and within these demonstrations, but the way this appears now to 
be set up, that will be much more difficult th^n I would like to see it. 
I'^l have some suggestions later on as to how we might change that. 

First, let/me try to convey to you the spirit with which these 
results are received in the Labor Department. It is not our objective 

"to have every. program get a positive evaluation. We do not want to 
have all of you tell us that everything works, because the purpose of 

<?this is' to find out what works best or what doesn't work. We'll be 
very annoyed if all these evaluations come in saying that everything's 
great. It'Will be very helpful to us if we find out that some things 
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don't work at all. We tri^d them and they were ideas that seemed 
promising, but we had to discard them. 



/On the research ^side, it's probably always true that the majority 
of research never pays off. You just don't know which proportion ahead 
of time. We have no way of knowing which of these things are going 
to work and so I think it's appropriate that we are trying lots of 
thijjfgs. That doesn't mean we want them all to be successes • They are 
in no sense all prototypes of what we want to do* We would be happy' 
to see some failures, a whole mixed bag of results, which will al^Low^ 
us to rank these in relative terms, I know I speak for my boss 
Assistant Secretary Packer and for his boss, Ray Marshall, who is 
very interested in letting the chips' fall where they will, in an ^ 
intellectual sense. " ^ 

A feature common to most of the demonstrations being discussed 
in this panel is that they are not experiments with random assignment 
groups dnd, the like. They lack the pure .classical experimental approach , 
in that all the results are hedged ahead of tim^\ ' Some of the researches 
sound a great deal like advocates rather than scientists. It may well 
be that advocates run better programs and that may be a model for program 
runners that we do want to tap, bi^t^it's probably not a good model for 
getting research results on something you might think you could repli- 
cate on a large scale. 

If I wanted to compare results across projects, what would I need? ^ 
I guess I would just look at the textbook ^^ay you do research. ^ First, 
you should state the goals of the program. The overall goals of the 
youth programs are pretty much the same. We want to reduce youth un-. 
employment. We want income gains to come out of this and variety 
of social things like reduced crime rates. ^ Skill development, school 
completion, and the like are almost inputs to these other larger goals 
that I mentioned, that is they presumably result in better employment, 
more income, and the achievement of the social goals. 

Since all t^f these programs pursue different goals, the program 
should list ahead of time what the goal of the program is, how the 
motjel pursues th^t goal and that should give you a very clear idea 
of what you can test, what you can learn from the prpgram. The 
hypotheses should be stated clearly. They tend to be fuzzier that I 
would like to see thenu-^If they are stated clearly, cleanly, and simply, 
then we have a much higher probability of coming up with something at 
the end^, a much higher probability of being able to compare results • 
across projects, than if what is being shown is whether a very specific 
kind of demonstration can fly. V/e need to learn about the general 
results more than the very specific ones. Sometimes we learn general 
results from running specific d.emon'strations , but they should be 
designed in such a way as to yield some insight on the more general 
results. * • , 

One place to start, then, would be with the data collection. It 
would be helpful to have the concepts that ^are used here defined in 
the same way for all .the projects. It would be helpful to be able \ 
to use the data that these E^eople^/col Xect for other purposes after 
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this particular project was done. It would be helpful to go in and 
find out why seemingly contradictory results came out of these projects. 
Therefore/ it would be nice to know that these people had agreed on 
^ definitions of what was skill training, what was work, what was whatever 
^ the concepts are that have to be defined. There needs to be prior- 
agreement on concept definition before the data are collected. There 
also needs to be prio/ agreement on categories of data collection, 
so that results caxy^e made comparable a'nd one huge data set can be 
put together encompassing different aspects. Ideally, we would like ' 
a very expansive data collection effort to be made for all of these. 
I realize that's not practical and so that all we can ask for, in some 
cases, where the data will not be exactly consistent, is that there be 
documentation of the data — extraordinary documentation. That is, how 
was the data collected? How were various things defined? What is the 
difference between your definition and definitions being used on the 
Mother projects? I think it would be nice if these contractors would 
share their definitions with each other and would sharpen up ahead 
of time any differences in concepts that they did have, so that we 
would know, after the fact, if we wanted to use all the numbers together 
^nd what relation they bore to each other. I would think an excellent 
part of the final project report #:)uld be a very precise documentation 
of data, so that it could be replicated and linked to other projects. 

As for research methods, 'I perfer to start with the classical 
experimental model and then ask, "How does one pj&)ject differ from 
another? V7hat did we do differently from the classical experimental 
approach — which is the way we're surest of our results ^ that we 
know exactly the probability with which we can accept* or .re ject 
hypotheses? How does your project differ frdm^a classical experiment?'* 
Many'of the discussants have already hedged their> results carefully, 
saying that we cannot expect too much. - Yet it would be helpful to 
have those hedges stated in a very preci^ way and to link them' to 
what the classical experiment would have been. "^"What intellectual 
compromises have been made and why?" The answers to that would allow - 
us much more easily to compare the kinds of results that we get from 
each project. 



heard a great deal of comment about our. being very modest 
)ectations of all this. Well, for $100 million, we can't 
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be t6o modest and we do have to expect somp results.* If our science 
is Such that it can't yield precise results to subtle questions, then 
we're going to have to ask. simple questions. ^ 



It's much 'more useful to get good, solid answers to simple ques- 
tions than it is to get fuzzy, ufyreliable "hunches" to hard questions. 
Many researchers have said today ttvat we shouldn't place too much 
.burden on research, asking the imd^'ssible and coming up witK a bag of 
smoke. Maybe we really should c^mcentrate on the simple things that 
we can answer, rather than ^dQJopf about asking questions we know we 
can't answer. 

^ ) ^ 

Let..i\\e, say that I am irinpressed with the variation within these ^ 
projects. There are a goe^a many things being tried and I think all. 
the bases have been touched an.d most of what should have been tested 
is bei^g tested. I think\hat's exactly what was needed. It's just 
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that this incredible variety cpuld yield a bewildering set of contra- 
dictory results ; unless some consistency is imposed by us at the top, 

GENERAL DISCUSSION : 

MR . JONES I I think we may be overly concerned about the lack of 
classical experimental design. One criticism of the classical 
experimental design is that it controls for type one and type two 
errors but there may be a type three error — which is asking the wrong 
question when you might have asked the right question. We may be 
trying- to find increasingly better ways of doing things that probably 
should not be done at all! 

MR . TALMADGE :^ I think a wrong assumption is being made by various 
members of this group — that it's easy^o do good^ research, or at 
least it's easy to do good research answering simple questions. Going 
back to the comment about the classical experimental design, I think 
probably everybody in this room has heard of the Hawthorne experiments. 
Those were designed to address a very simple question — does changing 
the lighting in work area affect productivity? Of course^ they got 
very strange results. Every time they lowered the amount of light in 
a work area, productivity went up and it reached an all-time high 
when* people were functioning at a level of full moonlight. I don't 
think that very many people know that this particular set of experiments 
involved a control group. ' It turned out that every time change in 
performance was observed in the control group. Every time the 
treatment group's lighting changed, the performance changed in the 
control group, whose lighting was not changed. If we had implemented 
the classical experimental design and simply compared the two groups, ^ 
there would never have been a Hawthorne effect. 

^ It's clear that we can do statistical analyses and reach con- 
clusions, but a great deal of thought neeos to go into considering 
possible alternative hypotheses for assembling diverse kinds' of data 
to build a case. 

MR. BRIGGS,; I feeLwe're missing what is ^perhaps the greatest con- 
tribution of these jirograms in the discussion over classical research 
design. It seems to nfb that the major overriding focus behind all 
this is changing institutions .7 Programs are effective if they can 
cYikhqe institutions in desired directions. The apprenticeship system 
has changed a great deal because of apprenticeship outreach. Vocational 
education changed a great deal due to competition from CETA. What's, 
really missing here is concerrf^fof institutional change in the most 
important institution of all affecting humans — the educational system 
itself. Many of our schools have not done a very^good job, particularly 
for the youth served by these new program^. In some sense-, this leg-^ 

sislation has come about- at exactly the right time. The schools are ^ 
in trouble right now. Enrollments are declining. They're running 

, out of people. They're receptive to new ideas for the first time and 
ready for us tp move in. 

On the. other hand^ as I've moved in and studied at the grassroots 
of this program, *just going in and talking to the scHool people and 
seeing what they do, I find that there is a tremendous amoiiat of things 
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being done in schqols that the literature doesn't even mentioFk. The 
new resources have allowed some of these schools to do things that 
they could not do before because budget cuts were taking away their 
staff, their pepple, and their initiative. It's encouraged them to go 
out and do some entirely new thingg^that they've never tried to do - 
before. I think that finally, after 15 years, we're addressing the 
most important institution of all in this whole power establishment. 
If we don't get the schools straightened out somehow we're going to . 
have manpower problems 'forever. 

There's been a good deal of talk about uniformity or getting 
uniform measures of program outputs or program objectives. However, 
I think there may be a confusion of means and ends in a very basic 
sense. I think, when you consider the whole range of employment 
training programs, the results of the research and the demonstrations 
are used by a number of different actors and, depending on wh'ere those 
actors sit, they have different agendas. I v;Duld go as far as to say 
that I think that a typical legislator would probably think more in 
terms of whether services are actually delivered than in terms of 
subtle assessment of impacts. 

The distinction between ends and means, is very critical. One ^ 
program that is a very good example of what I'm talking about is ^ 
Entitlement., If you look at the legislation, the ends are really 
"defined as keeping kids in school or getting them back in school. 
There's an implicit assumption that this is desirable and ^there's 
no mandate to test that assumption. . I think this sort of phenomenon 
occurs in other areas as well. I think there's a real problem because 
the people who use the research may not have in mind the same broad 
objectives as the people wl;io are doing the research. 

MR. POULARD: Perhaps there are some things we don't knqw that wa should 
learn and, upon learning them, midfh't apply them seriously* and honesty to 
address the problem of unemployed youth.. But I wish the solutions 
to the problems that have been raised were that. simple. I wish I 
-could believe that it i3 the absence of knowledg.e which explains those 
problems. 

I find it difficult to believe that ,the high incidence of unemploy- 
ment amongst youth is explained primarily in terms of what we don't know 
I have worked for a number of years v;ith populations for whom these , 
programs were created. As a practitioner, I can say there isn't that 
much mystery, as might be suggested, with further discoveries in 
uncovered truth.^ J wish that were the problem. It would be easy if 
the accumulatioif of a few more facts would provide the remedy. But 
the residual of so many basic societal patterns and attitudes, political 
stances and the like, seem to be so obviously at the heart of the matter 
Any serious effort to .understand unemployment would also have to address 
tY\e reordering of priorities that ^re not gracefully acknowledged and 
addressed. I do respect the fact that when one goes to the Hill with 
empirical data that is hard to- deny, regarding the cost 'effectiveness 
of one program or another, perhaps he does get armament to sustain that 
program. But it's more armament in terms of rearming a B-B gun as 
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m opposed to pulling out the cannons to deal with the enemy or the 
problem. I don't know that, by coming up with cognitive reasoning 
and new empirical data, we will address the attitudes, practices, 
orientations, and commitments, or lack of commitments, that seem to 
be the x-factors that explain the historical problem and the ever- 
increasing problem of unemployment among youth. 

T wish that in looking for answersydesigned to help remedy the 
problem of unemployment among youth, that. data found might be 
presented with pas*sion or that research engaged in might be done out 
^ of commitment, so that there is not merely numbers to surface,, but that 
th§re is support for a cause. I wish there could be "advocate research. 

I don't think that bastardizes research at all. It tempers it. 
It suggests perhaps a likelihood that the efforts of the researcher 
might better foster the objectives of the Department of Labor--and 
Ii don't imagine any researcher would have engaged in a* contract 
unTess those objectives appeared, even to the pure scientist, as being 
worthwhile, so that a case might be made for, for want of a better "term, 
advocate research, without denying the integrity^That the scientist 
- wants to hold onto. 

I would hope"t:hat it is not so much numbers and abstract charts 
and printouts, but rather commitment to the resolution of a problem 
that is factored in this as much as anything else in the process. 
Practitioners like myself don't have much to contribute to the formu- 
lation of a question that would provide the kind of answer which is' 
better or more easily collated in the computer, but practitioners who 
are worjcing in areas addressed by the Department of Labor do want to 
say to the scientists, "We hope that you find ways to utilize whatever 
data that you do surface, to advocate as best you can with that data 
the causes that the Department and the legislation and the Department's 
"regulations do uphold." 

It is too risky, too hazardous to just assume that it is appropriate 
let alone judicious, to take the pure researchers' approach. If the 

attitude behind the process is one t^at is devoid of passion and' com- 
'mitment, that is not a virtue • 
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MR. LERMAN; There's much to learn about what's happening and what 
has happened in the youth labor market* We've started to identify 
some of the important questions that we think are critical. They're 
broad questions, but we think we're moving in the direction of finding 
answers to some of them,^ and a year from now we'll be in an even 
better position. ' - . 

The first question that I think is critically important is to 
ask why the racial differential in"" youth unemployment* worsened in 
,tte last decade or so and especially in the employment population 
ratios. We see and have seen successes in equal employment efforts 
and even successes in narrowing wage differences among black and white 
workers. Given the fact that the situation was bad enough in the 
beginning, ^it is hard to understand how it's gotten considerably worse. 

The second question concerns the causes of such a generally 
high youth unemployment rate in our naAon. In European countries 
the relative «and absolute levels are much lower. Is turnover really 
the problem that people say it is, or are there other problems that 
cause us to observe, even in tight labor markets a high youth unemploy- 
ment rate? 

The third question concerns the seriousness as a social concern 
of youth ^unemployment. How is the unemployment spread amoung youth? 
If it v/ere fairly evenly spread, presumably it would be less of a 
problem^ How does it affect the current income of^families and the 
young .person's experience later on in his career? . 

Some answers to' these questions would cause us to be less concerned 
about youth rates and some more. Without going into any research find- 
ings, I'd like to point out a few interesting facts that demonstrate 
why further study is needed. Fot instance, it is frequently observed 
that youth unemployment is very short term in nature. This conclusion 
rests on data from Employment Earin^gs, where it shows that 4 0 percent 
of thef youth unemployed are unemployed less i:han five weeks, and 
another 20 percent between five aiTd 15, and only 'a small percentage 
on a long term' basis. That's true ^t a point in time, but if you 
look at how much unemployment youths experience over the course of 
a year, you * find that most youth unemployment is borne by people 
who experience r^ore than 15 weeks' unemployment; As a matter of fact, 
something on the order of .75 percent of all the weeks of unemployment 
amoung youths were borne by those who had at least 15 weeks or more 
of unemployfifent during the cour.se of the year. 

A second flequent statement is that when unemployitient rates are 
relatively low, the people that experience a lot of the unemployment 
are youth. While the ^.incidence of unemployment ^ high amoung youths ^ 
most Qf the long term unemployment during low unemployment periods is 

* still borne hy adults^. It turns out that tl^ group that made up the 
highest percentage of long term unemployment is prime age adult white 

-males. * " ^ ' 



If you look at the unemployment numbers that come out of ^PS, 
which are the basis for the 4 0 percent estimates of , black ;teenage 
unemployment — you find that these are not comparable with. numbers 
generated from other sources. I looked at about five' or six sources^ 
from various surveys, all with relatively similar concepts yielding 
yery, very different results. " 

MR. WISE: We have been looking at several different sources and 
types of information to first try to identify the dimensions and the 
characteristics of youth unemployment and then to see if we could find 
factors- that seem to be contributing to or simply associated with 
the unemployment. We've been looking at, cross section data, some 
time series data and some i>ongitudinal data. This means that we* re 
looking at Current Population Surveys, Census data the National 
Longitudinal Surveys and another longitudinal study put out by the 
National Center for Ecjucational Research. 

The first -binding is that we get very different results on basic 
things like the unemployment rate from different sources, namely. 
Current Population Surveys versus longitudinal data. For example, if 
you compare results based on the National Longitudinal Survey with 
Current Population Survey data, it turns out t^at there is a month, 
October of 1972, where one can make a rather good comparison. 

You find, for example, that the unemployment rate for white 
males based on CPS data is 11 percent.- That's high school graduates, 
by the way. The comparable figure from the National Longitudinal 
Studies is 5.4 percent. For blacks and CPS gives 23 percent versus - 
' 13 percent from the National Longitudinal Study. 

We don't know the reasons for this. It's not simply a matter 
. of definition. Maybe it has something to do with the fact that one 
is a household interview where the questions are asked of the head 
of the household and where the other one is a questionnaire ansyered 
presumably by the youth themselves. However, the dif f epenceax^eemed 
to us to be much greater than could be accounted for by the/aif f erences 
between the .questions that were asked. 

It turns out, not suprisingly, that there are substantial varia- 
tions across areas. The variations tend to be associated with 
demographic differences across areas; that is, the proportions of 
the labor force who are youth tend to be related to aggregate demand, 
which is no surprise, rate of grow.th in the local economy, and the 
movement of industry from central cities to suburbs. 

Anothei^ finding — I should ,say a non-f inding--is that there is no 
evidence that high school training — industrial-vocational trainings- 
has anything to do with earnings or employment after participants 
leave high school. It does turn out, however, that persons who worked 
^through high school seem to have much better experiences when ^they 
leave; that is', not only in terms of wages but also in terms .of how 
much they work. / " 
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It turns out also that unemployment, as one might expect, is 
linked to^ other social problems like broken homes, families on welfare, 
violence and crime. For example, we conducted a preliminary survey' 
of youth in the Boston area. We had a student go to^ basketball courts 
and interview people v;ho were playing basketball. It turns out that 
the first i20 people interviewed indicated that they had committed a 
nonviolent crime in the past week^ 

These suggest the kinds of things we're exajnining, and I would 
like to indicate some of the possible implications. It appears that 
aggregate demand policies can reduce the youth unemployment rate, 
but they are not going to so^ve the problems of the youth that are the 
^hardest hit, such as blac,k-^gh school dropouts. It also seems to us 
that policies need tQ^be directed toward this group; that is, policies 
directed toward solving the youth unemployment problem fn, general are 
*not going to do the trick for the. people who are thp most severely hit. 
And then one has to^ think about bringing these youth into the labor 
force. Most of the'm are simpl$^ not in the labor force. 

What about research priorities? Well, it seems to us that we 
need information on youth that are not in the labor force. Current 
information is simply not ve^y informative about people who are complete 
ly out of the labor forcer-^VJe need to know how they use their time, 
how they support themselves, what they are doing in general, how they 
'get jobS'When they get jobS;. because eventually most people move into ^ 
the labor market in one way or another. In that is implied that we 
need to have research that is focijsed on this rather narrow group, 
although it also needs to have a broad perspective. 

And then one final thing. It seems to us that there's ^more that 
must be^done to relate problems of youth unemployment to personnel 
policies of employers and other demand side practices. ' ^ 

MR. TOIKI<A: 1'^ going to summarize two studies for you. The first 
one is a rather small effort which is nearing completion, which 
investigates the potential impact of employment and training programs 
on youth unemployment. The second one is directed at answering, the 
question of what is the social and economic significance of youth 
unemployment. In doing the first study my objective was really to 
look at the process in a decentralized .system such as CETA by which _ 
a federal grant to a CETA prime sponsor is translated into labor ^ ^ 
market impacts. In thinking throjixgh^this process I was able to ident-- 
ify 11 independent factors that come into play. Their magnitude 
surprised me. Some of them are self-evident; but what I attempted 
to do was to put all. of the factors together in an analytic framework 
which^ would allow us to say what the impact on youth unemployment would 
be of introducing a manpQwer program and leaving it in force. If there 
is no behavior change, if the people leave the program just as they 
came into it, then the program hks a small statistical impact on the 
labor njarket-, .depending critically on the targeting of the program and 
the job placement rates. I ^m in the process now of trying to calibrate 
the model to simulate the impact of the YET programs. The program 
can be effective in getting youth Into jobs, but if all that happens 
is that they're in dead-end jobs t/efore they came in the program and 
they're in dead-end jobs apter tl^y leave the program, there's not 



going to .^e any substantial long term reduction in the unemployment \ 
rate* It's fairly self evident, but I think the strength of this 
analysis is that it-allows the integratioli of researcli findings. 

Topics amenable to empirical research include fiscal substitution 
and crowding 'out of jobs in the private -Rector and the incre^^se in 
labor forbe participation ifates which may occur because local labor 
market conditions are made better by the program. When it becomes 
clearer what the empiricalT^agnitudes of some of these things are, 
fit provides a' method of seeing^ what labor market impacts will be. 

The second study, is larger an3 it's asking a very basic ques^tion: 
"How serious a social and economic problem is youth unemployment?" 
I think we'd all certainly be in favor of reducing unemployment, but 
tHe tough part comes when you realize what the cost of this might be. 

..For instance, suppose we faced the uneasy choice of reducing youth 
unemployment at the expense of increasing unemployment of other groups 
in the labor force. Even though our focus is youth, it's our feeling 
that the unemployment rate is an inadequate measure in many ways. It ^ 
.ignores discouraged workers — people who've left the labor force because 
they'^can't find jobs* It ignores involuntary part-time workers. However, 
one could make the reverse argument, particularly with the youth pop- 
ulation ,^ that unemployment may be less serious than for adults for a 

' number of fairly^ obvious reasons. For example, youth are, less 

frequently breadwinners for their families. Their non-market opportun- 
ities such as school or leisure activities are typically greatker than 
for-adult workers. '"And so wq approach this by trying to sort Qut the 
issues. Which groups are worse off, when you take ^a bfoader look at 
it?' You just don't look at unemployment, but must 'distinguish betAveen 
full-time and part-time employment, the reasons for unemployment, and 
much more. \ 

The technique of analysis that we're using is to take a March 
Current Population Survey and link it with an April survey so that 
we're able to observe chanaes in labor -force status that occur from 
month to month and really aisect the analysis at three basio questions: 
First, how does^ the labor market experience and behavior of youth - 
relate to family incomfe?''^"^^vioiasly the problem *is less serious if the 
.^unemployed are from affluent families than if they're from poor families. 
Also it's not clear what the relationship between the employment status 
and earnings of the head of the household is on the labor market 
expe^riences and behavior of youth. Thefe's some reason to feel that 
young people come into the labor f9rce because of hardship, because ' 
th^ head of the household is unemployed. 

The second question is how much of unemployment i§ truly voluntary, 
in the sense that ydung v;orkers ^shop around for and frequently leave 
jobs? How much is the result of employer decisions or layoffs? 

The third questifin is that of the longer term implications. 
There's a ve^ry sharp correlation of age with both labor force partici^ 
. pation rates and unemployment* rates . Participation rises with age, 
and unemployment falls. If we could be absolutely sure that that were 
true for everyone, the probl^j would really not^ be that significant.-. 
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it would take care of itself. While teenage unemploymeni: rates are 

high and unemployment rates for the older groups are smaller, the 

labor force participation rates of these older people are larger, 

so there are more people in the labor force. If you actually 

translate that into the numbers of unemployed, you find that there 

really are substantial numbers of unemployed, let's say, in the 2 0 

to 24 age group as compared with the 16 to 19 age group. It*s entirely 

possible that there is a set of young workers that are continuously 

unemployed, and if that's so, it^-s certainly a much more serious 

problem than if unemployment were randomly distributed in th^-^population. 

MR. ZIMMERMAN; The purposes of our stuSy are, first of all, to assess 
various strategies ^or estimating the value of the output of partitipants 
in youth programs and, secondly, to provide at least preliminary inform- 
ation on some of the estimates of the- value of that output. 

We are goirtg to 12 prime sponsors, and we also are going to _one 

Department of Interior YACC site and one Department of Agriculvture 

YACC site. We have essentially completed the field work and now are 
in the beginning stages of data analysis. 

We've found th^t the prime sponsors, like probably a lot of 
pepple 'in this room, appear to have very mixed feelings about whether 

'Or'ndt^ this whole endeavor makes sense. Some think it's not very 
important at all, especially those who really believe, especially for 
youth programs, that we should be concentrating on the longer run 
benefits — whether they' be a return to, or continuation of, education; 
or whether they be earning gains. But some think that it's yery 

^important, partly because they believe there is a high correlation 
between the value of the output that's produced and the efficiency 
with which that output is produced and longer run measures of success 
of the program, and partly because they believe that it's extremely 

' important to them politj-cally 'at the local level to produce real outputs. 

I think ours is essentially a data collection study, getting down 
to the project level to pick up information about the value of the 
output produced. I t^hink I can't go very much further without addressing 
the question oJ^ what is valued. 

In the most basic sense, the value of the output is usually assessed 
in terms of the supply price. That's the price that an alternative 
supplier would have charged to do the work. 

But the Aext question adds further .complications. What is the 
demand for the output? Is it a service that is useful in the sense 
that there is a measurable demand for it? How does one try to assess 
the usefulness of the ^work that's being done? 

We want to find out, whether or not one can select projects with 
-"relative ease, whether or not one can get estimates of alternate 
supply prices and Vhat the problems are, whether one can get at least 
information about the demand for the product or services that are being 
delivered' and then whetlier one can go further and talk about a quant- 
itative estimate of that. We also are getting into project costs in 
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order to estimate the net value of output* 

There-are some theoretical issues that I think we will be, able 
to clarify a bit. We want to know whether this work is being done^, 
efficiently. Yet one problem that we've had is the treatment of 
inefficiencies in the regular work force. For instance, in one case 
where kids were doing maintenance work in schools, there was never a 
question about whether or not they were doing the work efficiently or 
whether or not they were putting in the hours. The problem was that 
they were doing work that had been done two or three years ago by the 
regular work force and that the regular work force was not doing 
additional work over and above this. It was not a problem of fiscal 
substitution, because the regular work force was still being paid. 
It was a matter of having a little bit. more t<ime for coffee breaks and 
that kind of thing. Is that situation an indictment of the youth 
program? It. is hard to say that, because we .also found cases where 
people had been put into situations that really did free 'up other 
people to do more skilled work. 

I think another problem that has- to be considered is the treatment 
^ of training time. One can dismiss it with a head in the sand strategy 
by only going after pure work projects, but that's not easy to do, 
because in many cases 'there are mixed , pro jects . - , 

Demand issues are an even more intractable problem. The fundamental 
issue, for me, is that youire dealing with what essentially is a,oi6b©unter- 
f actual situation. If you want to find out whether there's a demand 
for the work, you have to ask people- what would ^have been done in the'' 
absence of the project. Some people have a hard time even thinking 
in these terms. In several cases in our study we found a situation 
of people saying that if the youth project hadn't been there, they 
would have used Title VI workers, who are doing something else. There 
are a lot of programs out there, and the whold-^issue of the way publ|.c 
services are being provided and the sources of payment is important.^ 
We've found in some cases wh^re it is indicated that in the absence 
of the project, the worlc^ould not have been done, that if project 
funding ceased, the work would be continued from other sources. The 
project itself induced a change in t+iinking about provisions pf some 
services. U^fulness, thus, depends quite radically on the perspective 
and the respondent. . ^ » 

The last issue is whether we can streamline work valuation st^udies, 
perhaps by doing them without j3roject^by-pro ject analyses or, instead, 
with some standardize^ format. 

We have been unable to develop major short-cuts to date. There 
are possibilities for some kind of valuatioh mechanismjin a reporting 
system. There are two examples of that right, nov/: Oj>e is used by the 
Department of Interior and the other is used by tlm.'^partment of 
Agriculture. It appears that they might have a m(^el that could bring 
in some useful resuits. T^ie problem is that 4:t ' s a^-dnput based 
measurement system. They have various wage rates thatj they ,use by 
type of work, based (tffrNLhe--wa^s of regular employeesyqoing the work, 
and they simply multiply the waae rate by the number of hours that 
the kids were doing the work. 
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MR. ROSEl^: />I'd like to first discuss the measurement of work output 
of youth brograms because -I. think it represents a classic ^ample of 
an. attempt at ex-post facto "analys is. We rushed to get programs' 
"underway*, and then a year later we called' in a bouple of researchers 
and said, "What«Bid we do out there? Find out for us an^ tell us. what 
the impact w^s.'.! This is a similar situation to when we , first started 
looking at the results under the Manpower Development Training Act. 
We discovered 'that you can't usfe operating statistics. You're just 
kidding yourself. What you find is that nobody's measuring the same 
thing, and they're all using different definitions. You..ought to 
set up certain .sites that you consider your basic data site.s, and it 
is from those sites 'that you'll really get some hard information as 
to what you're really accomplishing.- To go in afterv;ards and try to 
pull out information from opeirating people who are trying to survive 
on a day to day basis, will^not be productive. The system is not 
designed to ' provide " statist ical information' of aijy validity because 
they're ask'ing this of operators who cannot answer the questions for 

USv ' A 

The theoretipal work in the "causes of ^outh unemployment evoked 
■ a certainMeja vu. I'm skepti(:al that the work wi^l h^lp to drav/ 
^^conclusions that will help in the formulations of policies direc.ted 

toward the employment and unemployment problems of youth. I don't 
' quarrel with the. data sources, nor with the methodology, but most of 
the questions you're puttrsuing we already know the. answers for: that 
is, we know what the statistical resuUs will be.- The question really 
is whether we .^ave enough guts or enougtv sen^e" to translate thesfe into 
policy issue^< • . ^ 

For instance, you talk about broken homes. What are we going to 
do about brqken homes? You're going to prove again that a lot of 
these kids come from broken homes\ thaK^ lot of these kids are° 
illegitimate, that they're from welfare families, that they're assoc- 
iated with crime and they*re a-ssociatgd with drug abuse. "What^then? 
bur difficulty is making the jump between the information that's 
available and the policy. The question is where do you move from 
those facts to program and policy and Who has the coverage ,to make that 
move. ■ ' ' • ' ' 




There's very little discussion about institutions and what Miey 
do to kids-. It seems to me that we ought to .think about -that rather 
'than just simply manipulating some secondary data. We must realize^ ' 
the fact that we have institutions that do a lousy job of what they re 
supposed-^ do, and unless you bring that in, you're goiTg to be 
leaving out some important parts of what you're trying to get^at. 

Simply saying that the institution is no good and then walking 
away from it or throwing mor^ money into it, is- not a sollition." We 
ought to be a littl^bit more, original about creating new institutions 
that pick up where these- institutions fail: We need to do something 
more about picking up youngsters , .for example , before they become drop- 
outs and having' institutions that' face the^ youngster as he- moves gut^ 



of the schpol system because the school system, is either wareho'using 
the people that we're concerned about! or pushing them out, 

MR> ADAMS: Having recently , completed a. book on youth unemployment, . . 

, I submit that the literaj:ure ' tells us a lot-ipore about the questions ' 
that are being asked than, has been acknowledged. In the theoretical ,v 
studies of causes and cures/* the objective is to determine the char- 
acteristics of the youth UQemployment problefh in the United Sta±es 
and ttje causes, of the f)ostwai: increase , ^ including demographic changes,, 
the proble^ms of tl\e central city/ the youth school to work transition 
problem, et cetera*. The'se explanatijOns *have been explored in countless 
other places, and* I think the knowledge^^to, be yielded by further work 
is ^limited. The most ^impoor'tant element woulcJ be any help -determining 
hovi^ jobless youth ^finance t'heif consumption and' spend.their time while 
outside the labor irarlcet. We^ simply kfV8w>very little about the non- ' 
market activities of *youth and the "alternative opportunities offered' 

,to these \ndividu^s and the incentive structure that exists as it 
affects labor market actiyiti-es . * * ^ - " . 



. I'm also encouraged by -the plans to, massage the data from the 
high school class of 1972 because,' ^'rankly, too little has been done 

fwith these d^ta. It/ s. almost been a mystery as to what the data have 
to say. Certai^nly ^the potential is there, and thd early findings . 

.illustrate the potential importar^ce -of this analysis. My hope is in ^ 
their further analysis they will shed light on at least r^ur problems: • 
One,'^^blackness 4s- a source or cause of the problem of teenage unemploy- 
ment; two, the j^ole of educati^oir and racial returns to .education; three^ 
the residual pffects of teenage joblessness; and, four, the nonlabor 
market pathologies of teenage joblessness ±11 terms of the incentive 
structures-. / ^- ' 

^ /Turning to. the study ^f the potential' i^act of employment and, 
training programs on youth\ unemploym^j^, I have doubts about- its ^ 
potential because it focuses on ^emple^%snt and unemployment , the 
dichotomous division oE /labor market activity. In so doirig i.t "'ijef lects 
3^*vi6w .that these "conp^epts measure something important. I'm not ^ure 
they do. ^ I'tMnk the -point ^was made that for some Who are unemployed, 
*the hardship may hot^ be signif iqan,t . At the same time some df^those 
who have- j'obs, may, in faat, have very significant slt&rt and lon^ 
t^rhn employment problems.. -Consequently, looking at the dichotomous . 
nature of tiie labor inarJc-et in terms of employment ^^nd unemployment. 
Status— r^a^^ly do^s-nft prpduce .v*ery , interesting result^'. 

\ r -> ' ' ''^ . - ^ ^ ' £ ' * . * 

In the study^ on', the , social and economic significance pf teenage- 
unemplayrnent , the ojijecj^ive is to measure the extent of teenage' 
unemployment*, the e3ft:ent to -which teenage linenvployjnent imposes hardship 
-on a teenager'? family, ^Ijie ^extent to which teena(^e unemployment arises 
from voluhtaory -supply d'icisions and the" ^tent to which teenage 
employment pfesa^efe of* subsequent^ unemployment . ' Tlie study focuS*es. on 
^horjiryfun labor' market dec^si6ns,^*matching March-A^ril CPS data 
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th4n can possib]|.y«,lDe delivered using*; the C?S.'. The * determinants 
bi>'r ^.market. statu^Hcondifeions .are being 'Explored by the study, 
ard not^ nor cfn -there, be' any controls, §or ^hool enrollment 



status. That's an extremely- important variable to consider in terms 
of the eJnployment or social and, economic 'significance of teenage* 
unemployment-. Most of the work which i-s to be done with transitional 
profcability models could also probably be done with tabular analysis. 
To me, it's a scalpel being used to dissect an elephant. Finally the 
one year work experience that the CPS will provide, in my^ judgment, 
will .provide vei^y. little useful insight into the residual effects of 
teenage unemployment. We peed to go much beyond that. In jjneasuring 
the work output of youth programs, the objectives are to provide 
preliminary information on the value of output of youth work projects 
and to assess strategies for valuing that output, I do not understand 
why, we are going to look for data, after the fact, rather than planning 
-ahead f or • thfe data.- th^t is needed in evaluation, -v Quite frankly what 
we ought to be looking at is not ^ the immediate good or service prdauced, 
but what it d6es 'for the individual, what the long term impacts of that 
progr,am are. That value of the goods and services, while of interest, 
is not- the most > importanb question. 

GENERAL DISCUSSION: * - * ^ 

MR . TAGGART : * In designing our knbwledge development agenda, we have 
to make ^ choice between dollar's for research, for evaluation and for 
demonstration. What is the best mix? Earlier, we were talking aboiit 
action programs delivering benefits to youth. Theoretical work does 
n6t have such an offsetting benefLt. 

How much theoretical work is-^it worthwhile doing? We must then 
subject theoretical work to ±^e same set of policy questions to* which 
we subjected ' the demonstration activities; that is, in what time frame 
will significant policy results be produced, what hypotheses are ther^ 
and on what grounds will they be accepted or rejected? Then, how much 

• of ^ basic research or empirical analysis should we invest in, when should 

• we invest in IX, when should we expect 'results , arid what track should it 
be on.? These ^re. the same questions that were asked about the demon- 
stration efforts. ^ ■ . , 

For ' instance, it seemed to us that work value should be estimated. 
We realized the difficulties, but "substitution" is ho easier to 
measu3ce, or guess at, than "work valuation." The questions are, 
"Are we researching" the 'right issues or the wrong, issues?" , and "Are 
w^ investing enough: do we" need to invest more?" The modeling simula- 
tions ^^ie/a'very cheap proces"^ compared to the other things that we 
,do. ' I'D design these demonstration project^ costs more than it does* 
to run 'everything that's* been done up here. What is the relative cost 
effectiveness? ^ . . • , 

MR. B^RNQW: I'm not sur? what to say abdut how much money one should 
put in, but maybe there is something" to be said about the ^sequence. 
I think if one were to look-.baef: over 50. years, at lea^t in social 
welfare research^, and ask what had an impact, there have been two 
'kinds of things. One is assessment of , problems . A second is demon-- 
strations of possible solutipns. You need both types «f work. 

MRn TOIKKA: This panel is discussing basic research with the intention 



of that research having an impact on policy. We must establish at least 
two. things: ^ one, that the research illuminates questions that have 
not previously been adequately examined; and, two, that that knowledge 
would fnake a difference in term^ of Prograniratic choices. I thi^nk the 
employment impacts of government programs have not been adequately 
explored and are a f undame^ital issue with obvious policy implication?.,. 
The same is true of the study of the social and economic consequences 
of teenage employment, ^ . " 

In many cases, researchers do not, for a variety or reasons, make 
the necessary link between their research and the policy options. I 
think this is^.not entirely the fault of the research communitj^. It 
grows, inZ-nty opinion , out of a lack of communication, the fact th^t the 
individl^als involved have different agendas, different priorities. 
It also grows out of the fact that there has been an incredible volume 
of research generated in recfen^ years, and it's difficult even to 
summarize that, let alone to make judgments as to whatf pieces of* 
research are better' than others. And, in many cases, I think that we 
rely on the judgment of other researchers to make those assessments. 
In some ways this is good. In other ways, ho\yever, it is unfortunate, 
because I think it leads to a narrowness of focus, * One way to expand 
focus is to f acilitate^more communication between the research community 
and the people making policy and .administrative decisions. 




SESSION V; DATA COLLECTION AND EVAUJATION 



PRESENTATIONS : , ^ ' 

MR. BRANDWEIN ; We've set up the national sample survey tracking 
system -to aid in evaluation of the overall CETA decentralized programs 

Very briery, we use the census to collect data on a sample of 
enrollees, selected, from 147 sponsor samples at the end* .of each 
quarter, We d-evelop a sample which is interviewed to get mo3;e de- 
tailed, more ^re\iable data on the characteristics and preprogram 

* experience of the enrollees, than are possible from sponsor reports. 
We .then follow up the same perioc^cally to determine post program 
•experience. . W^ aije developing a comparison^ group , using the CPS as 
the basis. and will use social security earnings data to compare the 
record of this group with that of the CETA participants to estimate 
the' net effect of the program. 

That's all very easy to say. There are substantial conceptual 
and technical problems with this type, of comparison group effort,, 
and I'm not optimistic about early ©r definitive results, so. we will 
be trying to develop some alternative ways of estimating net impact 
or the difference a program m^kes. 

In" any event, we've now extended the Continuous Longitudinal 
Manpower Survey (CLMS) system with no changes to cover the fiscal 
year ;78 enr611ees of" the tv;o YEDPA programs for which funds are 
aWLocated to all sponsors, YCCIP and YETP . 

The immediate purposes of this are quite Realistic cruite 
.achievable, subject to some only modest logistical ' ^nd timing limit-^ 
ations. The first purpose is to document who^^n fact is being 
served py the ^o YEDPA programs- inform-ation will also enable 

• us to do a general comparison of the characteristics of the youth 
enrol lees in other CETA programs, with those in these two special \ 
•YEDPA programs . . ' - \ 

The seco^ purpose is to get th^early views of the participant^ 
* on the program. That will be useful for public relations purposes, , 
if aothing ?lse. 



And the third pi\rpose isv to try to deterrine from foUow-up , 
tracking how long enrollees stay i^ the prog-ram, whether tlUre a*re 
significant; differences between those v;Q6 stay in longer and those 
who leave early; whether they rejourn tcy or rema^in in school,, and 
what tliei/* initial post program labor mnrKe-t experiences are in 
terms of^employmcnt and carninq§/. 

9' "Aoetin, it wil^ be possible to document any' major pattern of 
difference b^tween^ these tvc YEDPA programs, and between them and 
other CETA activities with large .youth enrollments. 

There have bepn three fssues in this extension. One has been^ 
/ whether and to what extent 'to try . to develop a^ psychological profile 
of enrollees as well ks the socioeteonomic one,* and whether ^ measure 
attitudinal changes after program participation. The Office of^outh 
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Prograim has wanted this:. I've ijeen dubious about it and the Census 
Bureau |[ias been, too, in part, because of some skepticism about its 
utility; moreso because of concern that the attitudinal questions 
would affect responses to other questions, would adversely affect ^ . 
response rates, and might have substantial biases, since we antici- 
pate that the interviews often will be conducted in the presence of 
a parent, or others v/ho might influence responses. 

tve've compromised and we will add such questions on experimental 
balsis for the -f iscal year *79 sample on perceptions a'boiit work, occupa- 
tional aspirations, self esteem, other attitudinal 'matters , using 
fundamentally, the same ones as in the National Longitudinal Survey 
and a number of other youth studies, so there will be som^ comparabilty.. 

We are going to present the questions to half the sample, and there- 
fore, we'll be able to test the ef|^cts of such questions on response 
r^tes. We will also reduce the risk of hurting the whole sample if 
the questions do -significantly alter response patterns and response 
rates^, . A ' 

The second issue is on timing of follow-up , We started the CLMS 
with a follow-up about nine months after entry, which is roughly six 
months after the initia^l interview, OUr purpose there was to naintain 
response rates and ease* recollection problems But on the negative 
side, the limited time before the follow-up interview has preant th^at 
many, or most, of the enrollees were still in the pr^ram at the time, 
of the interview, so that the' initial post program d^a'were for onl^^ 
the relatively early leavers. Vie are actively considering deferring 
the first follow-up until 12 to 15 months after enrollment, so as to 
get some post program information^ on virtually all of the enrollees. 

At the moment, we will have one Follow-up for/ sure on the YEDPA ♦ 
enrollees as well as perhaps a second. We'll judge ^fehat in light of 
the experience and findings .of the first follow-up. 

Finally, 'we have the basic issue of how you can ^estimate the ' 
net ii^pact Qf •YCCIP and YETP on the econom^Q experience of parti^cipants . 
The CLMS dat^ we're going to get will give us only gross information on 
how the 'participants fared after they left the program and will not 
ena^ble us to det'^ermine how much of their experience is attributable 
to program participation. Developing a comparison group for youth, I ^ 
think is more troublesome than for adults because .it ^s not feasible 
to, get a reasonable match in terms of prior labor force experience, 
because during the unsettled .teenage years, t|iere is . considerable . 
movement in and out of the labor force and in and out of sohool, V7e 
h^e the f urther, dif f iduity created by the scale -of the program, 
the programs have be6n expanded over a period of -several years, they 
c}o in fact, reach a considerable proportion of the low incorfie population 
to wjbom they are* . targeted , and^, ^so, some appreciable portion of your ^ 
coHiparisoij group' will at some point enter a program ra'ttigj than remain 
in'' the untreated category. ^ ' , ^ ^ 



, The size' of the .programs also meai\s that the labor markets are 
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;it*s fair to *3ay that the kids who areh/t in* the program, in school 
or out of school, are functioning in a les^ competitive larbor market 
at the time the programs- have enrolled substantial numbers of people 
and can't really be regarded as unaffected by the programs. - ^ 

Be that as it may, we will undertake some comparison with several 
types of reference groups. We^ll see what we can do by referring to 
the Job Corps comparison group, the supported work comparison group, 
and the CPS youth group. We may have some considerable questions 
about matching, but if the cumulative evidence from, a variety of groups 
adds up fairly clearly then I think we have a more solid basis for 
any estimates of impact. 

MR. BORUS: The National Longitudinal Surveys have been qoU\q on since 
1966 . What we are doing under the Office of Youth Prcgram funding is 
starting a new cohort. This will be 12 , 000 youfig men^and women, ^equally 
divided by sex, including a heav^^ overs^mpling of blacks, H:^spanics and 
non-black and non-Hispanic poor. ' They will be interviewee^ a^nnually for 
the next six years, beginning either' the last day of January or the first 
day of February, 1979. ^ 

We are going to 'have a number of background variables, since, in 
'the first year, we are>going to try .to get detailed information on what 
happened to^the young people, 14 to 21, up until the time of interview. 
So w^'re getting family information, schooling information details 
.on past training, military service, health and a little bit about 
previous, work ex'perience. *^ 

We're also including a number of socio-psychological measures in 
order to get some fix on what or .which of these measures may be good 
predictors of future behavior. We are including the standard internal/ 
external locus of control, a knowledge of ^he world of work scale, a ^ 
new thihg that we just invented , -which is similar to a.demangl curve 
that we use to determine willingness tCwork, the reactions of "signifi- 
cant others" to Various^ types of behavior, attitudes toward the role 
of women in the labor market, a"nd questions on aspirations and expect- 
ations.. 

Finally, in this first* year we will have what happened to the 
youth during 1978. There will be detailed woric history, which includes 
all of their major jobs for the ye^rs, including more detail on their 
present, or survey week, ^ job and its c'haracteristcs including such** 
things as their - satisfaction with the job, .^ome of the fringe bene- 
fits, and the quality of the/'work. We will get Irrformation on their 
and their faftiily's income for. 1978. We have a section on participa- 



tion in various types of tra 
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.ning programs, particularly DOL programs? 

wwwww ^^^^^ , .._il seek to find out what services they 

received and what the^ir reactions were to the prc3^i?-a«vs-T---We will get 
information on family stat-us, on educational expereinces and on mili- 
tary service duci>rfeg the "year.- . . 

We had' a lot. fnore that we were hoping to ask in this first inter- 
view, but our protest ran almost twice^as long as our 0MB approval 
'Vould allow, so that we have cut back our survey instrument: ' However, 
we ' have a number of questions for ^cceeding years. 
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One feature that we are tdstina for the second year that might 
be of part^tcular interest to people* is a tiih^-use diary to find out 
' how many young people, particularly those who are out of school, are 
, spending their time* We have also been asked by the Department of 
Defense whether^ we would consider adding a special additional group 
to the sample-iAyoung people in the same age g^r^ups who are now serving 
in the militai^y, siijce they are interested in binding out what ^happens 
to veterans when compared with youth who da not go into the service. 
We are beginning to talk to DOD about Adding approximately 1100 more 
individuals to the sample ♦ 

MR. WURZBURG: The National^ Council on Employment* Policy is involved 
in doing a pifocess evaluation of the implementation of YCCIP and YETP. 

Most of the questions that we are aisdressing in the. evaluation 
fall within the ten principles that were outlined in the 4^outh charter* 
We ' rsr looking at knowledge development. We're looking at quality work 
experience, ai^ what sponsors were doing to improve the quality of • 
work experience over pas.t programs, what improves supervisioji and 
what tangible outputs result. W^'re looking at youth participation; 
what 'roles the youths are taking other than as participants and what 
forms of participations are being used, other than the forms that 
were mandated by' the L,abor Department. We're looking at targeting 
to seis how prime sponsors are nsing their discretion for participant 
selection and what the; practical implications of the different eligT- 
bility standards /^'re. We're looking at the kinds of fiscal substitution 
that are occurring with YCCIP and YETp and we're als^o looking for 
different kinds^of substitution "^ther than fiscal substitution, that 
is, other things that may be diminishing the impact of the youth program. 
With respect to institutional change, we're trying to determine what 
effect YETP and YCCIP are having at the local level, on the relation- 
ships between prime sponsors, the schools, unions,* and private^ sector 
employers. We're also trying to figure out ,some of the implications 
of these changes. We're looking at the delivery agents, how they 
were chosen, and how they can be differentiated.- Then we're looking 
mor-e generally a^ some of the barriers ,to implementing the vduth policy 
at the local level — ^the systemic barriers, as well as some^£'the local 
Jjiistorical factors that have 'an effect in shaping the /impact of the 
youth programs. * ' ' ^ * • 

^' , ' * 
For the project desigh, we decided to take the ,case study approach 

for a number of reasi^ons." In spite " of what a* lot of prij?\e sponsors say, 

YETP is still essentially^ in a decentralized, decategorized program, 

with, a lot of variation. The program is new and issues are constant ly 

emerging, changing^ and evplving. ^We feel tnat tl^is is the time for 

observation rather than any kind of hypothesis ^ formulation and testing. 

The case studies are covering 37 prime sponsors. We have ten. 
field associates^, each one covering three to five prime sponsors. 
They're preparing three sets of interim reports as well as ^ final 
report. The i^nterim reports^ along with an overview goes to the* 
Labor Department. So* far ,-»we ' ve completed tw6 reports, one in Febi-uary, 
and one in August.' The third report is going to bis submitted around 
the first of the year', and y:he final report is .due next , Spring . The 
final report,. will be the'wrapup covering impler^ntation through early ^ 
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winter. ' ^. 

We've encountered some difficulties, inevitably in a case study - 
approach there is ^ tendency to look for generalizations and sometimes 
it's difficult to separate where the 'generalizations are .important and 
where the variations are important. Some of the field associates 
are also running ,in to a little birt of difficulty getting data. After 
thejL first two reports, some of >the prime sponsors are getting a little 
hit more protective of their turf, and they're not happy with some of 
the findings. 

In the first tv/o reports, we have .come up with .what I think are . 
some fairly interesting findings. Some of them may be useful in ) 
changing -the programs as they are operated .right' now; others will be 
useful in addressing reauthorization issues. 

« 

The planning process, which ±ook place last fall for fiscal 1978 
programs, for the most part lagged behind schedule. It was rush^ed, 
cor\fused and marked by a very disturbings^ind of dualism. \ 

There was, on one hand, an exercise v/here prime sponsors collected 
numbers for th^. grant application package which was submitted to the 

^ Labor Department, wh:^le the real decision -making v/as going on in a 
separate arena. T^e decision making process was not in any way reflect- 
ed >n the grant application package. The numbers used for the grant- 
application package were frequently seen as being inaccurate and unrent- 
able while other variables were considered much more important ia 
determining program mix and target populations. The grant application 
also required standardization which may have been useful to the 
Depar tment of » Labor, but was not particularly useful to the prime- 
sponsors. are recommending that the grantapplication package be 

reviewed and modified to get the information the Labor Department needs 

, without making any .pretenses of being useful for prime sponsors in 
their planning process. 

With respect to knowledge development, we found ihat j:ust about 
all -the prime sponsors • were trying something. Most of it appears to 
'be things that they would not havd been trying otherwise, but, the 
results have h^n mixed in the plans and the progress that we've "seen 
so far. While sbm^ have tried very structured experiments, others 
* are stifck4ng wi't(h simple variations in their management process. 

This year we are expecting that there* s;^oing to be limited payoff 
from knowledge development. In the first yea'r, prime sponsors had very 
limited capacity to give knowledge developm^t enough attention. They 
didn't have the time or the expertise, and f¥^quen£ly there was compQt- 
itign for the dollar'^s >that might go to knowledge development. A lot 
pf local pressure was put on priit^ sponsors to.keep^.the money in job 
^lotfe and in^ training. ^ . / ^ ^ 

. There is still a gre^t deal of confusion over l^^^knowledge 
deifelopment is supposed to be at the local level and how prime spohsors 4 
ar^ supposed to be using it:^. Some are tr^^ng- vefy .sophisticated projects', 
but, in atstemRting to do a' lot, ^ome of them may wind Up accomplishing . 
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very little. We've recommended thatNthe Department of Labor provide 
more technical assistance to both theVegionaL offices arid the prime 
sponsors. The 'regional offices ftave been found to be a gn^t source 
^of misinformation about knowledge development. 

We found that prime sponsors are putting an emphasis on work ^ 
quality. A lot welcome the emphasis on work quality, and they are . 
doing things differently. They are organising improving super- 

vision. They are paying much more attention t6 the quality^ of the 
jobs; however, not too much attention is being paid to creating jobs 
that are 'fully relevant' to the interests of the kid. Prime sponsors 
found it to Be impractical. They found that a let of the kids are 
too concerned about where their career's are going*. .. Instead prime 
sponsors have put a lot of attention in to more ..classroom oriented 
career exploration programs. We've recommended that career exploration 
activity and the idea of getting enrollees. familiar with different 
occupations be done in a classroom setting, rather than by on-the-job 
experience. It's cheaper and it's r^pte efficient. Jobs should put 
strong emphasis on simple ba'^ics—teaching a kid to show up to v/ork 
on time and teaching him good; basic work habits — the kind of thing 
that can be transferred anywhere. — . ^ 

Youth participation has^iigen very limited. The^ youth councils 
have been ineffective channels for youth participation for a number 
of reasons. Ki&s^' have not been participating very actively on the 
councils and,, the councils themselves are frequently ineffective forums 
f a2^ any kiUci of decision making. It's encouraging to find that the 
prime s-ponsors have tried a lot of alternative techniques for getting 
kids involved in ways other than just being participants. They've ' 
£hvolved them in some administrative aspects, they've involved them^ 
. in doing some evaluation, and they've involved them in peer counseling. 
This has been a much more effective rouJ;e and we've recommended that 
tfhe Department of Labor encourage alternative ^rms of participation 
without mandating any particular approach; ' . 

Most prime sponsors we've looked at have .concentrated^their 
services on kids whi> are the most economically disadvant%ed. Very 
few prime sponsors*, \f or example, are taking advantage of the 1Q% 
option under YETP to experiment with mixed income groups, partly 
because thfey have had difficulty with regional offices in getting *' 
research plans approved, but also largely because of political 
pressures to confine the prbgrams to the most disadvantaged kids. 
We did find' that ^females are grossly under-represented. We're -not' 
feure whether this is due t.o discrimination in selection or due more 
to the applicants* for the YCCIP jobs. Because most of them are i^labor 
/.ntensive, and a lot are in construction jobs, perhaps^ girls just 
^ haven't applied for them. ' ' \ 

-There's also some confusion about l^bor for^e status. We're, 
not quite sure who some of these participants ^re in'terms of whether*^ 
they were unemployed or whether th^ came from outside of the labor .i 
force. From what, we 've ' seen, most of them ar^e coming from outside 
-the labor fdfce, although the definitions are confuting. The implic- 
ation is^.of course, that anyone looking for the effect bf youth programs 
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in terms of reduced unemployment are not going to see an effect. 
They will see instead increased labor force participation. 

One of the concerns of the national of f ice, wa^Jthat expenditures 
for youth under Title I programs would be reduced. In fact, we found 
very little of that. Somebody really put the fear of ' God- in prime 
sponsors on that particular point, so there is very litt'le fiscal 
substitution of YEDPA funds for Title One, We did find, however, 
other kinds .hf substitutions over which the Labor Department had 
little control. Community block grant m^3*^y, for example, was being 
pulled out of some areas, and YEDPA money was bein^^. substituted , 
' We found that the schools, in some cases., were substituting money 

to provide services that they would have or shouldj, have, been providing 
otherwise. 

We found that community based 'organizations were extensively 
involved. We found it rather difficult to differentiate in any 
^systematic way among CBO's or between CBO' s and other services 
delivers, with respect to cost or target" populations , We did find ■ 
that the CBO's had the effect of broadening the base of 'community 
• involvement and the constituency of the youth programs, _________ 

As f ar ^as institutional, change goes, we think the CETA local 
educational agency links have born ^ome fruit. In many cases, however, 
the CETA prime spo^isors and ttie LEA's are not talking directly, and 
there's a lot of room for improvemen€. What we think is that the 
Labor Department ought to be looking for some alternative routes for 
« encouraging LEA participation, and for gettJLng the educational 
community more involved, \ _^ ^ 

MR , LACEY : After selecting about fifty youth work sites around the 
coXintry to look at, interviev/ing around /three hundred youths, and 
conducting about two waves of site visits, we came up with some 
hunches about v^at was^occurrinn out in| the field, But^ before turninq 
to those hunches r would like t6-%ive ybu a brief overview of th^-— 
research methodology MDC.used in our process evaluation. 

We were required by the Office of Youth Programs\to provide some 
specific kinds of data to them on supervision, attendance, discipline, 
and work activities, t'^e also were to provide some comparisons sup- 
ported by data and observations on the traditional youth programs 
under Title One as compared to YETP and YCCIP, The thing that w^ 
yjere attempting to arrive at here was wheth|*f YEDPA really mad|' a 
difference. We were also to provide assessments on \he kind ofi work 
that was being done from the standpoint of the commup/ity, the employer 
and the| yputh, > 

We interviewed youth, went to work sites, and interviewed prime 
sponsors'. We developed a . research ' instrument which gathered * inform- 
ation from the prime sponsor .about the kinds of activity that were , 
'^occurring ^ including enrichments, supported services, monitoring, 
^and supervision. From that/ point, we went to work sitBS arfd asked 
some fairly specific questions about activities. The thing .that 
I've heard repeatedly here, and that has probably been echoed for 
the last 10, years, is the issue of ^meaningf^l' work. We attempted 
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to look at different- kinds of work activity, •and really find out 
what is going on, if there are any tangibl6<Ba^oductS, if youth 
are jreally learning something; if their experiences are good. We 
were also interested in looking at these activiti?^ to determine the 
wages that youth were being paid, the hours, and the like. Lastly, 
we were, also to identify some exemplary efforts around i:he country, 
hoping that these particular efforts could be replicated by prime 
sponsors. We saw a lot oS kids involved in w6rk J:hat would not have 
been possible had not it been for YEDPA. Whether the work was meaning- 
ful or not involves a lot of subjectivity. ' We saw kids involved that 
would never have had a ^ob had it not been for YgDPA. «We saw good 
programs and bad programs — and in many instances, the good programs 
and the bad programs were within the same prime sponsorships. We 
collected data on characteristics, on supervision in terms of ratios, 
and we walke^d awey with a thought about V^at data: numbers are neat, 
expressive and pu^^r^but their ' simplicity^an be deceptive. We 
are very consciou(s o£_the limitations of the tabular and statistical 
data that we gatl 

Also, we looked at what ^we cailed good work sites, or identified^ 
some good work sites. We concluded that, first, work sites were good 
where the youth, as well as 'the supervisors perceived the work or . 
activity as meaningful. Second, where all agreed that some learning 
has taken place; third, where wages and working hours and conditions 
were agreeable to all; fourth, where supervision was a motivating, 
rather than a demotivating factor, and. fifth, where the youth perceived 
some future use for their experience. * 

These are things that constituted, in our minds, a good work site. 
While looking at these particular kinds ^of things we also examined 
whether there were conscious att^^ts by the^prime sponsors to somehow 
insure that good supervisioS^ would exist in terms of weeding out bad 
v/ork sites and helping superifl.sbr& jCelate to the youths. We wanted 
to knov; if the prime sponsor provided any kinds of training to super- 
visors. In most instances we found that they .did not. 

Also, we saw a lot of CBO's» involved in YEDPA, more involvement 
than in Title I. We saw QIC's, we savKU^ban Leagues; , apparently , hll ^ 
were able to get a piece of the action. • . / ^ 

Perhaps the most startling thing was the lack pf supportive .services. 
Youth across theJ country, from /Savannah to Pasadena, are laying that 
there are so many disincentiv^ to participating 'in t^ie program that ' s 
you find them terminating b^^use of costs., the need for transportation, 
dr ^fi^r^child care. The prime sponsors do not .provide these things.-^ 
You also find that public assistance payments, are sometimes being 
reduced because they participate in\ the program,^ causing some to drop 
out of the program. - * * / " ^ 

The researchers are now out in the f ieW for a third wave of vis- 
, its. We'rQ going back to the good sites, and tearing them apart, looking 
at supervision to determine why it's good. We are talking ^o th6 prime 
sponsoy: to see if it has made some kind of conscious attempt to provide 
.good supervision. ^ Hopefully, we will come out with some pointers on 
-howljko build a successful ^work site. 



MR, SEILER; .Among the lauhdry l\st of priorities of th,e Office of 
YouJ:h Prpgramjs is to have a^network of reviews of the proqesses of 
program/implejnentation that reflect on CETA^pAme sponsors. Now the 
programs I'm talking about a^-e not the special demonstrations , but 
the ones 'that ^are- financed' by prime "sponsors with the^ formula allocated 
funds — that is the YCCIP, YETP/ and SPED):, the Summer Program for 
Economically Disadvantaged Youth. 

Mr. Wurzburg and Mr-. Lacey haye reported on their efforts* at 
studying YCCIP and YETP, but we've'^also unii^taken some other contracted 
studies. There are several spe^cial efforts to ha^e our own .staff go on 
si^e to review sponsor progi?am' planning and operations. 

Undef YETP at least ^2%^of*'th^'^ funds^ must be spent on in-school 
programs where spohsors^'have'come ,toterms in the form of a written 
a^greement with local education /agencies; about how that program is* 
*going to be conducted. That prei^mably is a mechanism for trying to 
get sponsors and education agencies to better work together, to mesh 
>what they are doing.. In the initial Implementation of the program, 
it wasn'.t expected that they wofild have very sophisticated agreements, 
and they were given lots bf extra time to- develop these • agreements . 
We have had a staff effort tp look into those agreements and analyze 
them for their strengths and weaknessers as^-a means of giving-some 
guidance in strengthening *the agreements. We have ^al so had a joint 
DOL-HEW team examine CETAjjjLEA interactions in the field. We also 
'have a staff effort lookii^ at unjLon involvement in the youth programs, 
and there will be reports' oa thel'e; twp' things that will be coming soon. 

' -We've al^o launched^ a wide ranging effort to get a- bro.ad vi'ew of 
what the SPEDY program is'^all ^bout. First, we've had an analysis 
done on a national sample of the SPED^ plans which were funded. We're 
trying"^ to use those plans to draw a picture of what the program was 
supi)Osed ito look like, bcfsed on what those plans said. Another purpose' 
of that plan review is t6 look at ' the' acc^iiracy of the grant appli- 
cation itself as a structure fc?^ or as ^Rlanning tool for summer ^ 
programs. , ^ / 

Our ovm staff has studied a sample of primte ^sponsors by inter- 
viewing them early in June to get a pic5ture,>of the processes that 
were used to develop that written, plan, and to find ouf what were 
some of the underlying base? 'for their decisions that' were reflected 
in the plan- We then returned to the, same sponsors in July' and * August 
to study how the program was being delivered; to see it in action,., • 
.to look at the relationship to the original plan, and to ident:ify 
the reactions "and perceptions of youth about the program. As' an 
additional effort to learn about SPEDY operations. Manpower Development 
Corporation is looking at the character of work being performed by ^ ^ 
youth in the summer program. In addition, to get a better understanding 
of the SPEDY program, an organization called the National Child -Labor 
Committee has^done^a review for us of SPEDY implementation tq iden-^ify 
the kinds of dynamics and the .factors in the big city situation. In 
addition to that, as part of the National Council on Employm.ent, Policy ' 
study, there are participant observers in about 40 sitfes and we will - 
be getting some feedback on th^ summer program from tha^ study. JVe ' , 
are also ^studying the Vocational Exploration Program, which -.is. admin- 
istered through the J^ational Alliance of Businessmen ^and t;he'AFL/CIO 
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Human Resources Development Institute, as part of SPEDY. It operated 
in about 66 jurisdictions^ this summer, although there is some evidence 
that prime sponsors are not implementing VEP in the way that was planned. 

We are analyzing how. the VEP program was planned and implemented 
in two ways. We have some s^aff on Ni^B and HRDI taking their look 
and we have an independent study being made by the St. Louis Univeristy 
Center for Urban Programs. In addition/ we developed a special instru- 
ment for attempting to measure gome behavioral changes that might be 
expected out of the program; work knowledge, job search knowlexige; and 
attitudes and aspirations on the part of the youngsters. . We, deyelope(3 
a special instrument which was administered ^n a pre-post program 
basis within VEP to try to assess behavioral change. We are also 
trying to do a special analysis to -see if there are any .particular , . 
program organizational factors that tend to be related to any behavioral, 
ghanges we might find. Also,. we've had the same instrument administered 
within SPEDY programs, and we're going to attempt to do some comparisons 
of changep in summer programs, and the types of programs that sponsoi:s .' 
put underj SPEDY as compared with "VEP. Anotheif effort we had -was a 
national search for some SPEDY program models, and utilizing^ some of_ , , 
the perceptions of our regional offices, and otheglfe, and therq are 
five monographs being prepared on-' those programs. * , a ^ . ' * 

All that's being done on SPEDY is i,n readiness for 'our launching" ' 
of a series of regional conferences around the country on "SPEDY. These 
will tty to get some ear.ly feedback to prime sponsors about what has > 
happened in- the' program, and to give Jthe prime sponsors themselye^s an"^ 
opportunity to share that knowledge while it i's fresh;in their minds.. , / 
Hopefully, all of this effort will help. us improve the summer program 
regulations, the grant applications; procedures, the^ technical ^assis-^ 
tance program, and. ultima,te;.y , t)ie SPEDY design and delivery.' 

MR. BRi^NDWEIN: :Urxiijce the YEDPA^prqgrams. focused dn by most of thi^ \ "i. 
conference, th^/^ohCo^P ^^f^ '^ survived some 14 years of ups and dov/ns; ^ ; 
in public attitudes tqwa2?^^yestmen"t- i<n (disadvantaged yotitli. ;B\it 
^it ' s. hung 'c»/ and evolved, in tq ^a relatively stablg. progran\ with a ^' , * 
good notibn 3f whatsit 's doing.' Two .years ago^we decided to fpcu^on 
it^'for trfe p'rincipaf impact evaluation effort in the ydtith fieldr, ' 
bfecause/i^ was' a pr^igr^m, that^ Jtedneli^etf alr^eady;.' because^ it h^d,a ^ . 
sizabLe investment .f or ^participants di^d you could look for som^ ^* \ ' 
me^-e^ed res^ull^s;^ t^cau^e little was known about the magnitude ox ' , 
timing of its/eGpn6nuc**^f fe?^t ^9n participarlts; arid )5fecause we ha'd \ ' 
thought V^^had learned' m^re^'S'h-bvUt .the evaluation art than ^v^hen the*"^ \^ 
JdlD Corps\idiS a^^l^^t of so^e/^ontroVersial e^valuations in.-^^ 19€0'/'sl^^ 

Th'e evaluation focf^g^^qi ^bn ':^ive^typdi^^bf questien'sr ^Fi^fst, do^ 
,the J©b Corps participant's 'dc5 ^lett^ in/theV labor ^tria^e^t ^harr^<:ompar\ ^ 
able non-pa^ tiqip^nts/ St^cond, >.ow dcr^-they compare t^h'e ,n6n-^y/ '''^^;.><^^^^ 
^participants an-?ome othej: si^ifi cant** paramour sucih as refiwri^^to . 
school, , entry irftb military sery^*oe, '^;entry ' i,n^ *other^^^^ • 
degree of reli^ace orV welfare- paymfe^nats, aVi(J" c^&^ee'" Sr^^ 
, behavior, partjicu^^grly^arrest records and drpg^abv^g^^ Third/Vwere 
there marked d^Lffer^ces in effects by type- of' pa^t>icipant , ^duration 
participation , i,n ' the progir^m,' and by typ^ of* 3ob*Co*rps center?^ 
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Fourth, how diH the Job Corps participarvtS rate the program and what 
did they see as its principle strengths and weaknesS.es? Fifth, did. 
program* benefit^ to participants and society outweigh the program 
costs? 

Th^ comparison group was drawn from school dropout^ and 'entploionent^ 
service applicant li§1:s in geographic^ areas^whioh had i^Iatively ^low 
Job Corps participation, selected t)1ecause they we^e not adjoiniitg 
3oh Corps centers, and yet* in other respects wer^ sfmi-iar t,o ,the. 
areas frpm which most. Job Corgs e/ir6llees came. ^ ! 

In the Spring of L977, base line iriteryiev/s -wero, .conducted ;a?t 
the Job Corps cen.ters with a siz'able sample' of 'Uob Corps enrol lee s-r;^.^*-" . , 
about a third , of the enrollment at that -time.' "^h^.aampTe was made 
purposely large so that we could ,tr^lc*Dv^r a much longer period than 
the initial follow-up, *an4 y^t still have ^ sufficiently^ l^ajjge sample 
to di;aw some realistic conclusions. 'About nlne'^months , later, those ' ^ - 
who had terminated and .had been out at least ^ive months were. tracked 
and inteifviewfecS'. again, ^o obtain information on^their post program 
experience. That forj.ow-up sample was '2400. There was a" comparison' 
group 'iDase line' sample of 1500 with a. fqllow-up ,of 1300.;^ *, : 

And ear^* i:eport *on the basQ,, line interview d^ta .Wa-s puKlfshed. * ^ 
It describes*^he*'nature^of the Job Corps program, docuinents" 'tHe 
disadvantaged natyre of those/ it^ enrpll^, reviews "Wliy they saild^they , 
chose to enoroll arid Vi'esents^ their rat;ing§^f the proaram. ^. T^Fie ' ' 
follow-up s'tudy is not in firj^l f'of-il', bu^t softife/ import smt^fji^ndir^ . \ 

ar'e already evi<Jent. • ' ^ ^ " . I ' 'J ^ 

* On the average, the'pail'ticipan-ts did g'ain moi^e^ih emjpfpymeri^ . ' * 



ea 
we 



rnings'th^n ^the\ comparison^ 'gr^^up. The .prog-r'^* complet^r^^alA ,^fer^^^ » . . 
pdmp^^red^ lo* t^e c9mRari9Qn^ gi^Vg/' ^ ' ] ^ ' % 

^Ttj^ce-^'s an ijftteresting .time ^jJim^^^ifeni^he^e -In. the ^ij^tia^ Ao^ths 
out 'o|' tk% J'oh .^ori^^^ihe pai^ltici^ants \ag^^^ b^ind the pompariabp ^ 
,groJps. ^'T;ie"co5fi|5a;^^ group'^.had^i^ablei^urafa^r'^^ whp emijloyeA^at f 

Ithk *Bta^' oJ^ Xliat ^^fiQd , - while . i? t^ok t%mo tor- the ^Corpsmembers^ ' ' > ^: 
^tp** re sett 16^ and get^^irfSa jobsT Aftef the first tvb njonths, the -posit iv<j^ 
^ econoiT^-c 'iinggj&ts began ttp predominate. ^ Us^ng the ^week bef<S)rd th^. ' • \ 
^b3,*16w-up% survey as-the comparison poiiit, we f'ound *h^t ^a|'l of tfte 
^^ticfiparjts had gained $12 more "in vaverage weekly ,earnifigs tl^n the 
<56mpariVdh group ha,d.for the; s^me .petiod . . ^ .Font-he completers, it 
[ vias^$£3 more. On other measures- of Impact,. £oo, tlie ^completers- had*''^ 
s;positi^?e, 'larg'e, and 'statistically significant; benefits^.^ Mor6 were 
in ^the. la^bor force, more •were employed, they averaged more hours at 
•work, more were in military service more, had picked up. a degree in - 
thfe ^ Job Corps or by,gjDihg ba.c^ to .school ; more had gone on to college; 
they had less we^lfar^ dependency; '.and they had •fewer -^rre^ts and JesS ^ 
abuse of drugs. ^; , ' ^ ' ^ ; 

For those ^Jbo part i^a-ll^/ completed the p^o.gr^tt), such net impacts, 
were smalls In. ^fa^ct, . i^tie drbpolits iagged^ somewhat behind- the comparison 
group. ' " ' * " , \; .v * 



No sharp differences emerged at this point betv^en \ypes pr . 
sizes of centers. Seven months''af ter leaving Job Corps, 77% of - 
Corpsmembe^rs rated it, as pretty good' or okay, as against 61% who 
said so while they were in the , program. In part, however , I this is 
a courtesy response. Over 90% said bhe training was good, /or at 
least okay and fewer than 10% had some criticism of the training 
aspects. When it came to food, half-of -them said it was bad and^ 
there was some criticism of the relatively small pay allowances, f , 
Also, 60% said that after they got out they could have^ used more 
help 'in ' finding a job and didn't get much of it. 

Now, the 'final part of this study sought to relate benefits to 
co^t, and the report does develop some compi;ehensive estimates of 
the dollar value of ^various measurable benefits to par t j.cipants and 
to the society as.a'whole. At this stage, I think that part- of the 
report really offers more of a framework for analysis, an explanation 
of the types of estimating that have to be done. The end estimates ~ 
do depend significantly on various-assumptions about the continuation 
of rates of gain in subsequent years, ab9ut appropriate rates of 
discounting future dollars., and about the reliability of data- on 
societa.1 savings from reductions in criminal activity. But the 
initial estimate is that the dollar benefits to society for each 
Job Corps enrollee based' just on the first 7 months' post--program 
' experience, are 'larger than the economic costs of the program, though 
not by much, ' ' J 

The key limitation of the study, a r>ecessary or/e at this stige, 
is that its findings are based' just on the first s^ven months, so 
it's by no mean^s clear whether the participants' positive gains in 
the early period are maintained,, deteriorate, Vhi'ch is what has b^en 
built in to the benefit cost assumptions or whether they increase 
fuTther, which has been the ^pattern in the 2nd through th'e 8th month, 

* 

So we^will now carry forward this evaluation by following up on 
the participant and comparison group a year after the first follow-up, 
by which time we'll have "'an average of 19 months out of the program 
*from that sample. We '.re also going 'to track the base line interviewees 
who had not yet left .the program by the time of the first follow-up, or 
had been out fewer. than five months. So we'll have both^an enlarged 
sample of terminees and some check on. whether findings hold-up for 
participants who left the ^program at' a later point in time. About 
^ this time next year, in tim^ for the 1980 Congress, we should have 
readings on this somewhat longer term tollow--up. > ^ ^» 

I clpse by saying th^t without fussing about whether these 
findings are the definitive findings. for all time, we -have establish- 
ed that it is feasible' to do an evaluation, to do it on scale, and 
' to provide sound, meaningful evidence which hasn't been present before. 

DISC USSA NT: 

^ ^ * ^ 

♦ SUM : In the last y-ear,^ I've been a position of working with 

a goo'd inany prime sponsors in Massachusetts in developing YEDPA plans, 

r both by providing labor , market information and other information to 
help identify target groups, in drafting sections of plans, in * . 
reviev7±nq plans 'in a number of prime sponsor planning councils, and 
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•in monitoring som'e of our programs for the Governor ' s* Youth Employment 
Training Plan in Massachusetts. 

First, let me comment on the firyiing that 'planning in the first 
year was "rushed, confused and characterized by a degree of dualism." 
If any of you had been involved * in fflannirig at the local level,, what 
was said about YEDPA also held true for Title I, Title II, Title VI 
and STIP last year. Anyone involved in planning at the local level 
with Title VI funds, which will be withdrawn, if you don't begin to 
.spend theij quickly, is not going to place higher priority on spending 
a lot of time with your' planners and your administrative, staffiin 
(developing YEDPA grants. So I'm not surprised that basically we 
find that the process was rushed and c^Mifused. Most CETA planning 
last year was rushed and confused, and I believe that YEDPA was given 
a relatively low priority by many — ^jgiven the importance of pyj^ting 
Title VI money out in the field. 

* > -* 
But with the planning, process, there were two problems that need 
to be addressed and reviewed over the next year to see if we've made* 
gains. One is 'problem identification. Problem identification is 
generally weak in grant applications of prime sponsors. Previously, ^ 
we were talking about using CPS data to identify youth unemployment 
problems. But how would you like to use data base programs for 
Berkshire Ctmnty, Massachusetts, whei) the number of matdhes you've 
got is equal to about 4 persons? A basic part of our problem is the 
limitation of the localized data base. We don't really have a good* 
fix on local youth emplojiment and unemployment problems. The data 
base that we've got is v»y fragmented;' it has to be- pulled' from a 
great many sources; and it's not well understood many prime sponsor 
administrators yet. ^ . . ^ 

The problem , with prpblem identification is that it's not clear 

* who we should have been targeting our service on during the first 
year. I think we made a number of mistakes. In the ^first year, it 
seems in Massachusetts at least, that there was an overemphasis upon 

* out~of -school, unemployed youth. Basically what we found is that we 
had problems filling slots because we just exaggerated the number of 
unemployed out-of-school youth-^ we felt would be out there. 

What -I think is^ 'a worthwhile issue to examine^'this year is the- 
extent to which our experience in the first year led us to shift 
target groups during the second year. Are we focusing on certain, 
different segments of the youth population? Part of that learning 
experience may well be learning who your target groups really are. 

The objectives of youth programs are a second issue. Part of 
the problem with many of your youth programs is that there were 
poorly identified objectives in the first year of plans. The object- 
ives in most plans were more of a statement of ^intent to deliver 
services, rather than to produce various types of outcomes. Most 
YEDPA plans don't state objectives in terms of educational objectives, 
high school retention, job placements, etc. The problems with a 
failure to identify objectives ♦ in terms of outcomes is that it 
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hampers the ability of prime sponsors to undertak^e any type of 
systematic evaluation. If you're not clear what your objectives 
^r§/ how ^e you going to evaluate yourself? So knowledge develop- 
ment at^the local level is only going to take place if we begin to 
have a better 'understanding of the problems we're trying to address, 
and if we are more explicit within our plans about what;: our object- 
ives truly are. 

I see progress in these areas this year. More of the plans 
seem to be ?tating explicitly what the objectives of/ the YEDPA 
system are. I think that ought to be monitored fairly well to see 
if the prime sponsors are beginning ^o state objectives- in a more 
explicit fashion. . ^ " 

* Thei:'e^ is a relatively low emphasis on the preparation of youth 
for career" job^ by prime sponsors, and particularly low emphasis on 
on-the-job training and vocationally oriented institutional training. ^ 
The prirfary emphasis is instead on basic work experience, remedial 
educatio;!, counseling, career guidance and job search techniques. 
The question, though, is why? Why does such a low share of our 
YEDI^A funds get devoted to' those activi'ties? The explanations are 
quite Siverse and are' at times confusing. For example, it's frequently 
stated that youth are simply not ready f or' suth decisions. Well, ' 
basically^ the young age of participants is the thing that is influ- 
encing the services mix, and the question would be — do we find prime ^ 
sponsors serving* different age gr9ups having different mixes of services? 
Do, those focusing more on the older, out-of -school groups tend to 
have larger share of their funds devoted to OJT and institutional 
training? If not,'^hen how do we explain it? 

I wduld^like to know to yhat extent was the services mix for 
youth influenced by the services mix- under Title , I? Do those prime 
sponsors that run the larger share of Title I programs involving 
institutipnal'Skill training and OJT also run the larger share of . 
"their YEDPA programs in the same types of activities? I'd like to 
know, then, v^hether that's a learning 'experience? Do prime sponsors 
that have good Ti€le I programs transfer that experience through YEDPA 
programs? 

There's*^ a third issue around the question of how much of the low 
amount of training provided* to youth is due not simply to youth 3t- , 
titudes., youth preparation,' readiness for jobs and job trainiRg^ but 
r'ather to^employer reluctance to^hire youth in OJT programs or to 
hire them out of institutional training? Too often it is assumed 
that t^he problems are simply with youth themselves; when youth change, 
the unemployment problem will go away. What seems to me more often 
the case is that employer's attitudes and stereotypes have a lot to 
do with this. 

Finally ;»J^n regard to institutional' training, prime sponsors 
are reluctant at timeS to put money into that, because yoiir outcomes 
have to be stated in terms of unsubsidized placements. There's an 
unwillingness to devote money to, activities from which people are 
gbing to expect plax:ements at the end ot the pipeline. The way to 
be' safe in guaranteeingN yourself against condemnation by the regional 
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office is to put your money into work experience programs, because 
nobody really has expected much in terms of *unsubsidized placements 
out, of those activities. If we want to change it, we .might have to 
have the regional offices take i^a much more sophisticated look at 
■expected placement rates- out of those programs so that prime sponsors 
who are willing to take the risk- of putting more of their funds into 
training youth ^re n^t punished, ^ " - 

Youth Planning Councils, in general have not played^^ major role * 
ilj the YEDPA plans. For example, in the Balance of State, it's hard 
to get two people on a youth subcommittee to sit* down ^nd vote on an 
issue because all of them have conflicts of interest. . One of the 
problems is that^ Youth Planning Councils are dominated by the people 
running the programs. / The problem is not just getting youth to 
participate on councils; the question is how do you get councils to 
be active and play a major role in providing inputs into the system? 

The question of private sector involvement and how it can be 
achieved remains unanswered. How active a role have employers played 
in the planning and design of the YEDPA programs? if you v/ant to 
get the private sector "involved in YEDPA programs, then you*d better 
ri^^n institutional OJT prograips. Private sector representatives are ^ 
not going ^to sit down and talk about the greater glory of work exper- 
ience in the public sector. For the most part, they* re not interested. 
But if you want to get private sector involvement, you've got to ,get 
private sector involvement in the hiring and training of those youth. 
I would like to see YEDPA programs try to develpp over the next year 
tie-ins with the STIP programs. The kinds of e^nployers we're gearing 
ourselves to under STIP, the kinds of jobs we're gearing ourselves 
tQ, the kinds of wages that v/e're gearing ourselves to are the bdst 
in the employment training -system this pouritry has ever had. Is it 
possible^for youth to participate in YEDPA, go through that screening 
device, learn those basic attitudes, get that credential to be rated 
as stable,"^ and then be sold to employers in the STIP program-. I think 
tie-ins of that type are highly desirable; I'd like to see more o^ it 
in the future. , , ^ 

•I v/ill not say much about comipunity based organizations , 
except that in Massachusetts, a humbor of primes are drawing youth 
programs in house, because they are concerned .abou€ the fraud and 
abuse provisions in , the regulations. ? ^ . 

The experiences on LEA agreements appear to be quite mixed.' 
For the most part, they're largely non-financial in nature in my 
area. But I think even getting non-financial agreements signed^.is 
evidence of progress in this area. Getting the schools ^to understand 
what their roles should be and, ^^trying to gear their activities more 
towards students who are not college bound is a success. The question 
that I* would ask about LEA agreements i's to what extent are we mon- 
itoring them to guarantee that the services that were to be provided 
are, in fact, being provided? I would like to see* more involvement 
on the part of YEDPA with cooperative vocational programs and work 
study programs in high school. Cooperation with those programs 
has a number of advantages. They tend to have good contacts with 
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employe^^ They have worked out the problerRs* about' work scheduli1^g, / 
work arrangements, hour laws, aVid the like* Academic credit is \ 
given for a good deal of that experience* 

Op the substitutidh issue I 'm going to be a heretic here this . ^ 
morning* I am not as <^oncerned about the substitution issue as other' 
people are* It doesn't appear to be a problem* What I would like to 
have known is whether it^- varies by prime sponsor* This*year a number 
of prime sponsors are going' to ask to be given e^^emptions in the 
maintenance of effort, and I think some hf the exemptions for t^e , 
maintenance of effort make feense, particularly in those prime spon^r 
areas where over the last four years we never built up an institutional 
or OJT component under ^Title .I^._„'.When wa 're . spending„.6„5A. of ^ou money 
on. youth under the age of 24, one has to recognize that . it violates 
the equity provisions in terms ofj delivery of services to yadult 
economically disadvantaged* Wherj prime sponsors can justify . the fact 
that they have i^i the past been s(erving a substantially large Tmfaber 
of youth, they should be given some flexibility* -We ought to view 
it^n a case by case basis at. the local level* 

The last i,ssue is local evaluation* I believe that the reason 
why much local evaluation is not being done is simply that littrle 
subatafvtive local evaluation has been done for Title I programs 
where there are' more explicit and clear objectives, such as the ^ ' 

placement of people from on-the-job training and skill training into 
unsubsidized employment* Since we have generally not required / 
detailed' systematic follow-up studies of our participants ^n^Title I, 
how do we expect detailed follow-up capabilities to be developed in 
programs whase objectives are far more complex and more difficult ^ 
to measure, Buch as those under YEDPA? ^ 

I *think there is potential for changes i'n local knowledge develop- 
ment* During the last few years in Massachusetts we have been working 
with prime sponsors tp develop follow-up capabilities "for all Title I' 
adult programs* This year we're trying to bring in all Title I 
programs for youth* * Those programs have been supported by a cpm- ^ 
bination of state money, regional office support, university tie-ins, 
and some support from the national' of f ice this year* - 

V7ith respect to the expansion of CLMS to cover YEDPA iJrograms, 
I'm optimistic about the potential value of that, because 
•I thinkr it fnight help to provide a '^comparison group for those activities 
that youth are involved in under those' programs which experience a 
long term follow-up* For* example, we do follow up youth that are 
involved in institutional training, OJT or PS-E over a 36 month period* 
Now what we might be able to do is get a longer term follow-up to 
compare youth in wojfk experience with those going through regu,lar. 
institutional and OJT programs* ' - , ' ^ ' , 

What I would like to see considered, however, is th^ following* 
If work experience under YETP or YCCIP is significantly, different 
from' what it has been under:. Title I, then I would suggest, that we 
ought to include a longer term follow-up for regular work experience 
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programs under Tittle I to see whether or x)pt this 'different type of 
work experience has any longer term payoff on employment and earnings. 

» ♦ 

In the l^st year, Benriett Harrison arid jl prepared a paper for the 
National Commission on Employment and Unemployment Statistics * iji which 

^we made a recoipmendat ion that there be an expansion of ^the National • 
Longitudinal Survey for Youth, wi^th major focus tipon disadvantaged 
youth, to learn more about the mobility process. That's why I take 
a certain degree of personal satisfaction to ?ee that we^ have in 

. fact: 'agreed tO'fund such a study. "Yet, one problem wi-th the NLS is V 
the following The question is whether we ought to -"contiaue to draw 
on a national probabiJTity-. sample, covering most states throughout 
- the. country , or rather sTipuld concentrate on a few major metropolitan 
areas to learn, more about larger samples of'youth at a Selected number 
of sites. . ' ' ' ^ 

Now the results in the localized longitudinal studies will 
be les^ generalizable; hovever, 1 wou^S argue tha€ I think they could 
be more rich and more informative about how youth beKave in the labor 
force, and what determines their "success in the labor market. If you 
want to answer questions about the family, 'the*schools, and the labor 
market, they're best understood in a localized environment. So, I 
would, have liked to have seen a National. tLongitTudinal Survey that 
was geared to a selected number o^f metropcflitan areas to give us a ^ 
ri^:her understanding of what rea^lj happ^s *to youth. ^ 

On the questionnaire design, I would have liked to see in the 
new NLS more inf ormatiSn on the characteristics ^of establishments 
wheVe yc|6ng pegple find jobs. We have a tremendous data base on the 
' characteristics of "young people, on t?(e characteristics of their 
^ families, on their human capital- attributes ; we have almost nothing , . 
on tha attributes of the establishments in which they find work. 
For instance, recent evidence suggests that continued tenure with 
employees for black youth does not have the s^me payoff that it 
does foi; white youth, in* e.ither skill development or wages. Why 
is it that staying with your employer do^s not matter as much for 
black youth? \ There are a number of alternative explanations. foj that, 
but one explanation is that black youth end up in establishments that 
have substantially less market power, less integrated internal labor ^ 
markets, fewer opportunities for moving -up on the job. The question 
is whether or not the only way we're going to get Employment ^stab- 
ility to pay off is to pu;t^ black ^outh into different types of 
establishments. Knowing more about that, I think, will give a 
r4chear^story for people trying* to design empldyment and training 
poMcy. . ■ 

The last thing I woul'd have liked to have seen is a sample of 
yduth participating .in vocational education- programs throughout, the 
country, to see how vocational education programs work, relative to 
' the .employment training pragirams under YEDPA. 

Concerning the Job Corps evaluation, my dnly comment is that 
^while the aggregate analysis is useful for national policymaking, I 



would .be more interested -in results for each vocational area in order 
to dettermine the best training occupations..- * 

GENERAL DISCUSSION ; . • ' , ; 

MR. WURZBERG ; I would like to comment on the notion that kids should 

be matched with job6 somehow *tied to l;heir cai;eer interests.. It's 

one thing to talk after the. fact about .a* kid liking a job because 

this correlates with his or her cai^eer interest; it is^ something else 

to try to determ^fe beforehand what an enrollee'or a potentiaj. e'nrollee 

career interests ^are and try to set up jobs accprding to that. We're 

not recommending that they de-eij\phasiz^ work ' experience . But we're ,j 

recommending that they de-emphasize v/ork experience as the principal 

vehicle, for finding out about other careers. 

t - ' ' . 

A r. TAGGART: One of the really interesting things in the MDC study 
Is that It seems to indicate on the average that the new jobs under * 
YEDPA are better than the old jobs under Title I in terms of ^ the ' 
emphasis on supervision and on the quality of w6rk^ The way^ we pursu^ 
that through our regulations was to insist 1:hat prime, sponsors have 
high supervisor-tO"^youth ratios. And yet/ another conclusion drawn 
from the MDC study was that the Title I slots tended to have more 
supervisors per youth. But the supervisor-to-youth ratio regulation 
that we 4jad used to. achieve an objective was not correlated with the 
objective, iThe question was .whether putting it in the regulations 
achieved better work-site ^supervision even though there was no' 
correlation between the supervision ratios and the quality^ of^ 
supervision. That is, did we achieve' the objective by d6^g something, 
even though 'it had no direct correlation with the outcoYne/^ The ratio ♦ 
may be the only way to get to supervision in tfie regulations. It may 
then *be wrong to say thiat'ydu should not have in .the regulat.ions the 
supervisory-to-youth ratio, even if the studies find no relation to 
outcome. To judge from our MDC study f on the average, the new 
pr9grams had better work sites tha^a the old programs, what is it 
that made that happen? Tlje only mechanism that we had in the regul- 
ations was the supervisor-to-youth ratio. , ^ , , 

MR, LACEY: We found that the prime sponsor in a lot of instances 
had never even visited the sites.* Even though monitoring was stressed, 
there were good work sites and bad ones ^whether^-Ol^7i'tct^-there was tnon- 
itoring. I think there's a larger issue here concerning how you get 
institutional changes. People do t\\e right things for the wrong ^ 
reasons. And I , think the prime sponsors, a lot of them, looked at' 
the letteXof the regulations and reinterpreted it' in '•their own v/ay. 

Now,' what's crlrtical is how the regional office reacts to that. 

In some cases, regional offices took the . regulations .and' applied them 

very mechanically without thinking ah)out the objectives, I think a* 

lot .of prime sponsors are inclined to say, okay , . we''ye" nol: going 

toy get that • specif ic ratio, but instead we're dping sbmtethin"g else, _ 

wMich we think is qualitatively a. better way to get at that objective'. 

^ ' / • «. , 

«* «i ' » ^ 

MR, L ACEY: I think . you give them a simgle objective- that thfey can - 

hang FTiat on; a simple ratio; and then y.ou also build flexibility 
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^into t-he regional offices so that ' they don ' t enforce the things 
mechan ideally. • • • * 

MR> STRQMSDORFER: In assessing institutional change and the relation- 
ship to government policy I have rarely se'en an' awareness that when 
you do a process analysis, apart from monitoring the system as care- 
fully as p'ossible, you're going to have to look at things which policy 
can-affet:t and things which policy can't affect. 

-There are some variables or some policy tools which can be exer- 
cised at the federal level and some which cannot.^ Those are rarely 
.identified . v.*If yo^ don't clearly distinguish between those things 
which are amenable ta^policy and those v/hich aren't, the process 
studie? ajre^ ^oing to be sterile, and the government is left with the 
only tool il: its disposal'^or iTistitiational change, just dumping one • - — 
hell^of a lot of money out there. Presumably, we're trying^ to create • 
some sort of directed institutional change, and not just any old change. ^ 
I have the impression that the "studies so far have not focused on what 
sort, of change could be accomplished within^jXe current system.'* ' 
/ . 

The second, point I'd l^ke to make_is to get back to an earlier 
Comment made by Mr. Sum, that the-NLS should be targeted to few 
areas. The purpose of NLS is quite different from' the purpose of 
> other kin^s of case studies. You can have case studies but you also 
need information abput how youth are doing across the nation. That 
is the kind of information on which national congressional 
policy is going to be made, not on the basis of specific cases. I 
don't know if we have the proper balance in case studies, but I do 
not ^Eeel the NLS , is misdirected. • v 

MR. DIAZ: ^ We're just beginning our work site evaluation on 

Entitlement, so we really don't have anythi^lg developed on v/hether 
or not we have better work sites or not. It would seem to me, ^ 
thou^jh, if they are better 'work sites, that my *sense is that the 
primary reason would be that there are very large dollars* involved 
in the program,, and it was stressed from the outset that the quality 
of work sites would be followed up by an evaluation. 
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CONFERENCE SUMMARY AND A LOOK* FORWARD 



MR . HAUSMAN I . There ar^ three questions we should address in seekinjg 
to summarize these discussions: First, 4^;hat information needed 
to answer the questions which Congress is' in'terested in answering 
be^fore it re-*enacts youth legislation? Second, will the research 
that currently is initiated under YEDPA yield the numerical or non- 
numerical answers to those questions? ' And^)^ third, what programs ' 
either ought to be 'expanded or newly developed to deal with the most 
serious problems of youth unemployment? 

' -> ' ' * * • 

I think that the Qonference has yielded for a few of us some 
perspective on our research-*-some .recogjiition of ^the lim.ita.tions.^^ _ 
I have a sense that ''some peojJle have s unusally -high expectations 
about what research might yield. For example, I have heariv repeat^ 
references to controlled social experiments, and I guess I would \ 
wonder whether there ever has been one. ^Thus I don't know *that the,, 
youth program should be held to that^ standard when, in my view, and 
I think in the view of many others, it ha's never l^een done before. 

PR ESENTATIONS BY "CONFERENCE PANELISTS : ^ . 

t 

MR.. NICHOLS : , I don't have the list of specific Congressional , 
questions. The law spells out specific kinds of programs that had 
to be run and different kinds of approaches that v;ere to be tested* 
Their comparative ef rediiveness is really the 'big question wHicn 
■must b© addressed. But the information that's needed is information 
of the kind that I think we are going tq get from the Entitlement 
Program, though not necessarily within the deadlines. We must 
build the calehdar into t\ie planning of research, but n,ot promise 
unattainable research' results . On the Entitlement Program, We , 
should knoW how many people went bagk to school or are expected 
to go bapk to school. That's a simple kind of question.^ What is' ' 
the impact on local unemployment rates? That's a little more 
diffifcult to answer, but that's ultimately a Question that Congress' 
has 'in mi^ — can Wse, reduce unemployment *^and if so, at v;hat cost? 
,This is ^ question that looks for simpl/e hypotheses that can be 
.janswered, given the state of the art. We cannot demand definitive 
•answers to questions that science is not up to answering, but this 
is one" issue we must address. 

Conqress' qoal is to reduce tho unemployment rate, and that 
requires rcducinq youth unemployment now. \\vcn if the jobs pro(3rcims 
do not provide skills pr output value, they serve a purpose at tho 
present time. I think youth programs are useful — they are a source* 
of income. Also, in simple terms, jobs must reduce youth unemployment 

Vie once had an economy where four percent v;as full employment. 
We now have an economy wnere it seems we can't get near that without 
inflation stirring up. We ask Why this is the case.. The answer 
everyone gives us is th^* the structure of. the labor market is 
changing. It's not too 'big a question, ^1 think, to ask in v/hat way 



and how that structure has changed. If we can t§iy in what way it 
has changed, I think that's the. first step of knowing how io 
unchange it. - • ' , » 

. I tiave rto. simple answers, though I do thinTc the research * 
coiranunity has the responsibility to define' questions more simply, 
than they've been defined in the past, 1 recognize problems of 
nonhomogeneity that exist in any of these programs we're involved 
in, yet|I think we have -excused ourselves too long by saying our 
job' is dif'ficult. There are some things that can be done^ We 
mlist move towards greater -^refinement and specification, 

M^R . ' TAGGART ; Withi^n DOL, we have debated the research ^questions 
to determine which were prominent and highest-priority issues. We 
^ stated priority issues in the' 1978 Knowledge Development Plan. 
We have now revised and augmented them in the 1979 plan. In fact, 
we have anofher $215 million of discretionary activity, much of 
which is .already' underway, and the plan is soon to be implemented. 
.VJhat has surprised me is how little cognizance there is of this 
broader framework even arrtong researchers involved in par^s of the 
activities. The questi9n we have to address is not just whether 
our little part works or whether we can make it more effective, but 
how we bring that totality together. What's potentially different 
from wiia-t^we did in the '60s v;as that we began by stating the 
issues > and then structuring agtivitiea to address them. There must ^ 
t^e -cognizance, of structure if synthes^ is to be achieved.- 

Over the last couple of days we have had arguments * about pieces 
that should not be assessed in' isolation. Let's take the simple 
issue of work evaluation.. We -have four different assessments ongoing 
of work valuation. We have a' theoretical assessment of v;orK valuation 
methodologies'. We .thus . should tiave a number of wa^s of getting at 
these issues, and each must' complement the other. 

* • 

So the question^ are: What are the magical questions and what 
is needeSsto answer 'the magical questions? V7e haVe a continuing 
revised 3^st of wh^t are the major issues, and we have a set of 
knowledge development efforts that are directed to'* those specific 
issues. Will research solve those issues? I' think they'll solve , 
it as much*as research, evaluation ^and demonstration can ever resolve 
such things; 

^ There wefe a number of challenges to us to improve the content 
of those activities. Many- 9 f th^se are. valid. Others have been 
addressed.. The plan ?or next year lays out a whole set of programs * 
which fill many of the gaps that have been identified.* What are 
Icids in the* ghetto really doing with their, tirre? What should be 
done for special groups— -the handicapped or young mothers, for instance 
A^'variety of activities- to address such issues are written into the ^ 
plan for next year because we already know we have to address those 
issues. . ^• 

So somehow, everybody who is doing parts of this has tb grab 
hold of the totality. We have to be concerned not iust with what 
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we're doing but what everybody else is doing. We've got to coordinate 
einpirical and theoretical research with evaluation and with demon- 
stration activities* "We've also got to integrate institutional change 
measures, 'econometric measures, ^economic variables, psychological ' 
variables* -We've got to have pure research combined with advocacy 
research^as well as publicity efforts* • All these things have got 
to be integrated* • ^ ^ 

• Th'fere seems a reluctance to get to the-^ig policy issues, to 
get to this synthesis • Everybody seems to be focused on narrow 
que s^t ions* . " ' ' 

1 think the direction that ASPER suggests i's Crucial* First, 
what a;re the big issues, how ^e we going to solve them, and in 
what time frames? It's most appropria^te and .necessary .JEor„.everybody 
here to also 'address those same questions and also look at the big 
picture* And 'I think thei;e ' s, absolutely , no question that if 
everybody does that, we will hav^ the resources and the flexibility, 
to actiieve the major things that Congress wants* 

. I think what we have to do is look at how the total effort "^is 
structured* We've got to adjust our plans, if they are. deficient* • 
For instance, we did not^lay out the time frames for results or 
provide for their integration in the initial drafting of the 1979 
plan, Appropr;Lately , w^ were sent back to the drawing board to do 
that. , V 

As we go along, we're going to have to adjust and refine these 
plansr"! S^le're going to have to go to each evaluator and say, "Can 
yo'C>. speed) up on this, can you get additiona'l kinds of- data at this 
time-, 'be<^ause it's what we need?" And all of us have to do that 
together. ^ . ' 

MR* O'KEfSFj]; There are some very significant questions that Congress 
wants answered in the Youth Employment and Demonstration Act* They 
want to know what the effects are on educational participation, ^ 
youth income, increases in employment, and changes in labor force* 
participation rates* And I think that everybody in this room has 
'to Aake an effort over the next six fo nine months to pull togeth'fer 
whatever information we can* . 

.However, I think part of our responsibility ^ is not just respond- 
ing to some very narrowly-defined questions. Part of the process 
is th raise for the Congress and for the public-^at-large the other 
issues that we should be concerned ^bout as we pass a law--a law ^ 
that's probably going to dictate our youth policies fpr the«1980's* 
For instance, I'm not sure that yo^th policies in this country' 
should be directed solely at reducing current youth unemproyment* 
We do know these kids are going to grow up, we do know that they're, 
going to be in the labor' force 'for the next 45 years v^and therefore 
I think ev^y policy that we put in place should no^only look at 
its immediate effect in terms of unemployment or labor force 
participation or education, but it should also' ask the ques^/ion, 
how i^ it affecting the longterm prospects of these individuals? 



; If I had to give "an example of the kinds of questions* that 

we're being pushed to answer right no.w, I would say that we've got 
to find ou,t over the next few months how the programs affect. the 
labor force participation rates in narrowing the differencials that 
exist between minorities and whites. How did they affect income 
derived from^the employment? But, I think that assessments .of 
unemployment changes are much more problematic. 

For instance, I think there's a real question for educational 
policy' makers as to the extent ,to which labor market interventions 
are able to ind\ice youth to return bat:k to pchOol. I've oft^n 
wondered why, in our education policies, we haven't been abl^ to 
^experiment more directly with paying kids to go to 'school, especially 
^ids we can identify as being at risk. • Jt doesn't seem to me that 
-that-' s -a-- labor mark-et interventioB but rather an educational ' 
intervention,. 

Tbe legislative cycles. will be very unyielding. If there are 
going to be- any big budgetary changes in the next- budget Q^cle they 
are going to be in tfie youth area. The Adm^inistra/iion ha§)to respond 
to -the inflationary pressures that it's confronting and what^ere ' 
going to find is that there are going to be decisions on both the 
Hill and the Administration about' cutting the budget. 

In the youth area we've got to'" be very careful about makihg ' 
a case for how much money we want to take in a categorical fashion 
for youth labor market interventions. ^ ' • 

Over the past two days, we've -been told about a lot of good 
research, evaluation and demonstration activity that '6 been going 
on. Rialtt now there seems to be a- lot o'f people out there re'-writing 
the various books of the Old and New Testament. However, a lot of 
policymakers and policy Analysts like myself never really did read 
the Books. We've always * looked at the Classic Comics. There i-sn't 
anybody right now pulling 'together the Classics ''Comics vereion of 
what we're doing. ' • * 

The time frames are. fairly narrow, and therefore, what we've 
*got to think about, as ea'fch of .you do yo^ individual si:udies, is 
how is it you can relate it back, to what you've heard here, to what' 
you' know that's going on.^ ^ * . , ' 

But', I think ie- must do more than simply resjJond to^ legislative 
objectives'. We must put on the table the extent to which the 
ol^jectives we had fbr the current program, are realistic- Is it 
realistic to expect that a iDillion-dollar! program is going to 
reduce or 3,eriousl^ cut into the problem that's been with u6 for 
20 years, a problem that we've, spent sofnew^re in the neighborhood 
of $50 billion on over 'the past 15 to 20 years? As we go forwakd, 
we've got, to try to decide the extent to which our expectations for 
interventions in the labor market on behalf of youth are realistic, 
and whether those expectations can be carried out in -a system that 
pulls on a string at tl^ national level and expects responses at . 



the state *and. local level. 

MR. BRIGGS: If I were a Congressman, I would ask first abbut the 
impact on unemployment. I think the law w^ passed largely as a 
reflection of concern about youth unemployment-. But I hope that 
that'.s not the first question someone will ask, because' I think 
that would miss the whole contribution of this legislation. I -don't 
think the Act and^the programs will /-likely have much impact on 

■youth un^ployment. That 'does not mean that t think anything* is 
wrong with the legislation. The question is, "What is 'it we expect 
to ^et?" Is it job creation? Is it -reduction of unemployment? 
Or is it getting people back ii^ the schools, and back into learning 
situations? I think the prim^e sponsor's ee^ the latter in this 
legislation, and I think this is an accomplishifient . Entitlement 

.Programs and .a lot of ^local efforts ^strongly emphasize getting 
people back in tl;ie schools, or to stay in schools, whether these 
people were unemploj^ed or, out* of the labor iorce . So', either they 
were at school to begin with, or they were, potential dropouts 'and 
are' now being identified sp that they 'don't become dropouts. 
Therefore, it's going to be pretty hard to figure oui^ just what the 
impact might be on the unemployment statistics. 

.\I'f you 'really try to focus on out-of-school youth, thpse in the 
labor force fuj-l-time, then the problem^ is, of courvi^e, that you\- 
encourage people to drop'out of school, and that's what keeps the 
prime sponsors from doin$ it. ^I have run right into that in a 
number Of areas. One of the. reasons they don't want to sweei^^n up 
OJT fbr, young people ds that they fear that ^it ^^ill be an enticement 
to leave school. > And so*th^,y want: to stay away'* 'from offering tpo 
goo5 an enticemenjt. . ^ 

There are some mixed feelings in these discussion^ about whethe 
or not- the purpose is to get people*' b'ack in the schools or into 
educational training, or. into a. learning env^Lronment of some sort, 
or whether we want to actually get them into jobs right away. 
This paradoxical- question runs through a* lot of the legislation 
and a lot^ of the programs; the prime sponsors are faced with the 
same .qfuestion. ^ . ' ' 

I think the knowledge development activ.ities that are being 
conducte(3 ^focus on two separate themes. There's one theme of 
^ohsorin'g research on youth in general to learn sothethiTng about 
youth. It is marvelous that we're finally able to get attention 
.to youth problems' and yoHth labor market conditions and to identify, 
who the youth are and to g^sess the effectiveness of institutions 
that impinge "upon their lives. On the other hand, we also^Jiave a 
lot of activities trying to evaluate, study, and assess programs 
that have been created under this legislation. We have to ke^p 
those separate sometimes. I can support ^he basic research, the 
idea of doing basic Veseajfch for young people in general, and I 
think" that's very helpful, a-nd positive. I' 

On the other hand, I think we also*hSve to be careful ^that we 
may not set Che same standards when we get within the programs. 



The programs run into a tremendous amount of institutional ' re- 
sistance in trying to bring, about change, as in the CETA prog.r^m 
last year. Quite frankly, in most of the prime sponsors I have to 
deal with today, the YEDPA is a backburner 'issue. These, people are 
nov; concerned with private sector* initiatives at , the local l^vel . 
Since we were there last fall we've had'Title VI buildup, -the HIRE 
and ST-IP programs; etc. The most important t;^ings in the second 
year are to. keep the xouth program at th^ local level 'and to see 
to it that the prime sponsors and some of the other program people 
take it as serious and have the same interest that 'they .had last 
year. 

I think that ^e educational institutions, in the schooling 
process are the on,es on which this Aot may have the- greatest 

,>mpact'. The educational establishment is massive and very hard 
to charfge^. By definition, an "in^tq^tution" means something that - 
doesn't change very quickly. If'you try to find measures for 
desired outcomes, you are not 'going to find observable change in 
three months or six months. .In fact, -I really fear that you're 

'going to ru;i into the story of the man who went out to -pick his 
flowers every day tp see if they were growing, and killed the^^ 
The flowers have just b^en planted, the programs have just gotten 
started. Now, it's important to track them and watch them grow. 
But ^e't's not demand totally definitive answers so soon after many 
of these programs have just gotten off the ground. r ^ . 

J'hat leads me back tb one of the other institutional problems 
vthat'we've got that affects all of these programs, the whole 
CETA system. YEDPA, like all CETA programs, suffers from a lack, 
of planning at the local level. .They're trying to plan; there's • 
a lot of encouragement to local planning. But maybe it's just 
endemic to the whole process that there can be very little planning 
for CETA manpower programs. The lack of national planning in many 
ways filters down to the local level. , ' 

Under YEDPA, there was great pressure to get started last fall. 
In most 'cases, it really didn't get off th6 ground vntiX well into 
the spring, and all the pressure to get it going only d^ipplicated 
the process^. The first six^ months wer^ chaos at the prime "sponsor 
level. About the time it did get off the ground, the private sector 
xnitiatives werfe the big things pressing on these sponsors. It's 
very hard to isolate exactly what impacts of these programs 

are^from looking at either the program data or even some of the 
case studies. Thfeire are a lot of other things going on. 

. At the same time, institutional changes are somethirig that we 
do need to look for. I think they are coming with tije schools, with 
the vocational educatipn establishment, with academic credit Jor 
work experience, GED progr^s in rural Mississippi* and the like. 
This legislation has been a wedge for introdiucing a lot of nev; 
ideas l:^ke alternative schools for sponsoring some of these ,and 
saving some programs that y^e^^e about to get cut out because they 
>/ere too expensive for local school districts. ' ^ 
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It's a grab bag of accomplishments, but in m^any ways I think 
that the Youth Act, by focussing so much of the attention on 
education, has taken on the institution ' that is most important in 
youths* lives--and that's 'the educational system itself, it's not 
, going to change the educational system entirely. It's not meant 
to affect all youth. But for many of the yo;ath that have become 
the clientele for many of the programs in the past, this legislation 
may haVe some impact. ' ^ 

I don't. know how you actually convey that idea to, a Congressman 
in any precise way, because in one comn^nity it's the vocational 
Educational esta^blishmefit that your 're dealing with, another it's 
the dropoiit pi^ograms, and, so on. It's ,a variety of individual types 
of accomp^lishments. ^ But I think" we shouldn't lose track of these" 
a^ccpmplishments and in the second year Ve ought to continue to push ^ 
the prime sponsors into experimenting more in trying to reach nev/ 
groups and to try to document in some sense some of^the' institutional 
change that's taking place, as well as some of the normal quantitative 
measures. We're asking a variety of quantitative measures. Maybe 
we ought to ask researchers to tell us a few stories, and give us 
spme non-numerical accomplishments and changes. 

I've seen institutional changes in the communities ±hat I've 
been to — major changes that can^ be credited directly tojthis Act. 
Someone can pinpoint this, and ^ Say this Act has helped us ^ake this ^ 
change. As an example, consider a work proqram in El Paso, 
Peo|)le. there have been ^ pushijpg for ye^rs to get it; with this 
lefgislation it came about, rfi Albuquerque, there is movement into 
a whole* new, lindi^try,^ the hotel industry, by the cooperative education 
community. Nothing ever touched this industry, the largest one in 
th"? city, and haw they're doing it. 

There are a lot of things that" this legislation can accomplish, 
an(^ I don't kn6w whether you can capture, them in the quantitative ' 
data, especially if their real impact is likely to be or\^ the ^ * 
institutions, and especially on educational institutions. 

MS. WILLS; I think to understand these youth programs we ne^fT^' 
sense of their history. There's a very smart man we have in 
Washington, Ray Marshall. He went dov/n to an island before Carter 
waar sworn into office and he figured out fairly quickly- that the 
on/y *new money that was going to be in the btsadget in the first year 
oy the Administration was somethingccthat v/as going to be packaged 

id was going to be called an economic s4:imulus program. Mr. Marshall 
/ad no illusions .that it would solve all the problems of youth. You 
ran't do that in^a one-year program as part of an economic stimulus 
^program. He seized upon the opportunity, and thank God he did, and 
we had the youth legislation. 

Another thing happened, though, on the way to this legislative^ 
forum. One of the^criteria they used when they were trying to put 
together that early package was to avoid, new pieces of legislation 
by using existing authority. It was correctly pointed out to 
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Ray Marshall at that time that he cDuld use Title III, the Secretary s 
discretionary authority • But there was a very basic and fundamental 
problem: Nobody trusteed the DOL to use the discretionary authority 
of Title III in CETA legislation, including myself *and the orqanization 
I represent • 

Interestingly enough, he could have done* it with one paragraph 
in appropriations legislation; he could have gotten all the money 
thajc he needed, $1,5 billion. Instead, you have lOQ^ pages of 
legislation. In the process of ' getting that 100 pages of youth 
legislation, a variety of mission^ came into being. Quite ^frankly, 
it was a plea; it was an act of desperation on the part of concerned. ' 
oeople on the Hill and within the Administration. ^ 

J • ^ 

Indeed, there is not only the, economic stimulus to put the kids 
back to work._ As J recall the statistics, the range ,was -between 
600,000, up to 1,5 million youth in this economic targeted population. 
And indeed we can serve that 'number of aeopl^ with programs coming 
out of the Department of Labor in a yea]* or a year and a half. The ' 
question is, how do we touch^all those people, and with what kind 
of progranjs? ^ \ 

^1 remember well, "because I sat there and talked with the lobbyists 
of the education community that pounded on the door and. pleaded with 
the interim task forces that. were trying to put* legislative agendas 
together. They wanted very badly tife, get an increase in terms of 
fiJnding for their programs so that they could begin to affect the 
lives of youth. They were willing tb admit th^^ ha,dn*t done a very 
good job ,4, They didJi't get it, because it couldn't be a part of the 
'economic stimulus package. ' • 

^ '^^q^er kind of question that came out,, a-nd is one of the reasons 
that there's so much Secretary's discretionary money, is, "how do 
you really target?" You'lcnqw there are inadequacies in terms of 
^he formula grant programs in terms of targeting. . Is there any 
possible way within the political process to' target the funds on a 
formula grant basis that's goi?ig' to makfe any difference to the kids 
in the .inner-city ghetto and out there in the rural areas? - 

. * Clearly, another missibn of that legislation is institutional 
change. It's the publicly ^ fin-anced structures that* were very clearly 
the target. If you bother^ to take the time to read the -committee 
reports and the conference committee^ reports , it * ^ there, and it 
is a plea. That's why you have the 22 percent set aside, I'll nevex 

^rget the day that I was saying that you^can't do a 22 percent 
let-aside the way you have it written because there are 1200 public 
districts or 30 in-^^a particular state and there are only 2 3 prime 
sponsors; now how are yoii going to mesh it? The point was that' the 
Congress, in an act^^of desperation, because we, representing • the 
institutions in the manpower world, and the educational world hadn't 
bothered to get ourselves together, decided they'd do it for us, , 
Indeed they did. 
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Acade'mic credit wasn't in the first draft of the legislation. 
Three differe^nt times it was taken out as a goal. The Department 
of* Labor did* not want academic credit placed in that legislation. 
As you ban see,' it is in there. 

So I think, that there is a very clear plea for a varied of 
missions, a. variety of tests that the knowledge development planning 
is attempting to address. By the way, I think Ray Marshall was 
right. He l^cnew that there would be continuing"^ money ^and that it' 
would nqt just last one year, and I think he's probably smiling 
about thati. . ' ' * ^ 

0 

There" are so many missions that W.t is necessary to keep an eye 
on all of them. For instance, J think you're going to have to take 

look at some numbers on female participation, for example, in • 
YACC and YCCIP . ^ That obviously has some implications '"in terms of 
*progran) design, whether or not you have a separate part of the title, 
for that kind of program. More than that, throughout that legislation, 
when one bothers to correlate that with the legislation going through 
now, you will find that the Hill is getting much stronger in terms 
of the lamguage used concerning elimination of artificial barriers 
and. sex stereotyping. I think w^ need to pa^ some s|jecial attention 
t<g the issues of the handicapped. That *s* just^a policial observation., 
The Minority Leader in the House next year, for this committee, is 
very concerned about the handicapped and I suggest you cover yourselves 
on the issug. Besides that, it's an important •issue . 

• I think a question that's going to have to be answ^ed is 
whether and by what means we reach the' target populatiTDn,. which 
basically translates into;blacks, Hispanics and lov;-income whites . 
What are the implicatiojt\ST for instance,, of using the 70 percent 
lower income living standard? Should we raise or lower it? 

I 'Would also suggest that we v;ill have to consider universal 
coverage, responsibilities.. For example, there's the vocational 
education •program and all the 3ducational systems. I'm hearing when 
I'm talking to labor market people that kids are dropping out because 
of the real and/o^: perceived opportunities in CETA of the income 
transfer payment program and the family income and participation — 
in this part of CETA and the rest of CETA. What does that really 
mean and what implications v/ould that have fo:^ dejigningr policies 
perhaps more strictly on the income transfer side, as opposed to 
the manpower side? * 

/ ■• c 

Another issue is labor market and occupational information* 
Congress got frustrated with the lack '.of response on the part of 
the federal fiscal agencies, and the lack, of coordination between 
HEW and vocational and CETA systems. It created the National and 
State Occupational Information Coordinating Committee ♦ Rest assAjred 
you'rle going to be asked to respond back to Congress ai)out what you 
have done with the mission ^of the MOICC and SOICC, in ^ terms of the 
development of the career occupational information systems. Also, 
the local prime*sponsor system, in 90 percent of their responses 
on viiat they want out of state'^nd federal government say more 
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information on significant segments of the population. They're, 
desperate for that kind of information. They don't have any h^p . 
and nobody ''s giving them any help on that. And if NOICC ^isn ' t ^the 
•right solution, then some of you say so and give them better altern- 
atives next time around. ' 

I would hope that one of the things that you I re going to be . 
able to take a look at in the case studies are ways that would make 
things easier. This is. really a plea, having operated some of 
these programs out there^. One of the responsibilities policymakers 
have back up here in Washington is to try to write a piece of , 
legislatio-n that will make it easy as possible to administer. We » ^ 
often don't do that and we end up having to have five million different 
administrative councils, regulations, and the like. 

MR. POULARD: What information should the research provide? I 
believe one major concern should be delivery institutions. Questions 
should be formulated, raised, and collated that address the mechanisms 
and the institutions that seem to be ef^ctive, as delivery agents, 
as operators of programs and advocates of the right kinds of policies. 
I -think any .research well done would, or should, acknowledge that 
even certain of the sacrosanct institutions that have been a market 
for the Department of Labor have -flubbed over the years. There's 
nothing to be seen in their participation at present that can 
adequately recommend that ^they be le^t untampered with. Conversely, 
there might be seen some institutions that have had to fight their 
way in. These might be identified, and the research finding used to 
support their position so that th4' no longer have to force their 
way in. I ha,d in mind, in particular, community-based organizations. 
Ideologically and geographically they are nearest the poor. Almost, • 
all of\hem camfe into existence primarily, if not exclusively, for 
the purpose of helping the variety of individuals who are chronically 
and structurally unemployed. _ . 

On the basis of that generalization^ of course, one can proceed 
to argue that they ought ngt have to rely on circumstance and 
pressure tactics to force their' way into meaningful roles. At least, 
as research bears out the useful role they're ^erfbrming at the local 
level, perhaps their task, should be made easier regarding their 
future participation. *. . * 

I would <:ite, as one particular example ,^'th« fact that out of 
the limited discretionary money ir^' the YCCIP program, $12 -million 
is being spent by discreti9D and ^^iven to HUD to pass on to ten 
community-based organizations. Some of you ^en her*e might.be ^ 
startled to learn about the additibtial millions of dollars whicj 
these ten community-based organizations parlayed by virtue of tl^ir 
small grants. For instance, the Watts Labor Council leveraged it's . 
money from' Washington with additional money from the state, the 
county, and the city.- It was able to affect ^>ery quickly the 
appropriate kind of working relationships with the union and that 
was aeeded because housing construction^jvas undertaken. It -was j/ 
able to train a large number of ^person? in the construction trades. . 
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As a result, there is a housing stock consisting of new construction 
and major rehabilitated unit^s. being produced at a pace that might 
embarrass our largest contractprs* lt*s there for the skeptics to 
see\ Beyond that, most of these are home -owner ship programs, so 
that in 20 years the people, who currently pay their "siotes , will 
own the house^. Where do you get that kind of bang for the money? 
And who's doijjiig^it? Not some sophisticated institution but, rather 
a ghetto-located community based orgar^izatio^ . 

I'll admit there are some that can't do that, but many can. 
My point is on behalf of those who can. That CEOs have to spend 
too much time and too much of programmatic energy fussing, fighting, 
struggling, and trying to get in. * 

^ The point again is that I hope the research will yield information, 
that will profile adequately the meaningful role that some of the 
CBO's are now playing and might play in the futuj^e. ' 

I would hope also that the research would yield some views of 

participants. Frequently I hear it said that there are many thousands 

of jobs going begging, ' that unemployed youth, and others don't care to 

accept. Why is it that unemployed youth don't want tl)e jobs that the 

newspapers advertise, or other jobs that are supposedly available? 

^ Might these youth tell you, if questions were posed to thert, what 

' these jobs are like, the kind of wage they get, which when received 

represents an inadequate payment for what is expected of them? 

I 

0 

I would suggest also that questions be formulated regarding 
the views of user-agents. What do some of these user-agents tRat 
get the subsidised labor think of people sent to them under these 
programs? j * * 

If you find out v^hat I think you're going to find out, I would 
not ask you t6 shift gears or wear a new hat as advocates and no 
longer be scientists. IJhe rest of us will do that. If you were 
to learn the views of some of these user agencies, I have an hypothesis 
regarding what you would find. If* you were tc5^ make public these 
findings and hold them, up to public d9bate, I think at that point j ^ 
the advocate^^ would then assist the scientists i^n pushing that 
issue on bBhalf of the chronically unemployed. 

MR . LOVE : . There's o3^e thing conspicuously missing in the debate: 
the- role of the private sector. We have heard talk of "increased 
demand," but 1 don't know how v^'re going increase that demand. 
Yox instance, there's a lot of ^publicitj^/ behind private sector 
hiring. But the HIRE program never got'cDff the ground. In the 
Labor Department, it's known as How Industries Refuse to Employ. 
So, if you 'rfe 'talking about expanding job demand, it's got to be 
in the public sector. .That's got to be where youth legislation is 
geared. The AFL-CIO, organised labor, ha*s been spending many years 
training youth, as, for instance, in the Job Corps. This is important. 
But the end answer is that there are simply not enough jobs. 
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There is also the problem of adequate wages. Everyone seems 
to agree ;that the minimum wage is fine for kids. If all youth see 
is working at McDonald's, is that going to inspire future prospects? 

The biggest^ problem I've found in the last ten years is that I 
organized labor was not contacted and involved in program development ^' 
activity. . You wait until a program is designed and, then say "sign' 
off 'on it-" That's wrong. Apprenticeship programs are the biggest 
training inechanism in this country. TWey are join^tly operated by 
management and by ISbor. You' are not going t9 get into that system 
unless you deal with labof. ^ • 

So, if you're going to' go out and do research, read the basics. 
Organized labor is written into the legislation, and we'r^ going to 
be in it. If not, we're going to rap up and down the Labor Depf!f&ment ' ; 
halls and even go to the White House: We h^e fought for this CETA 
and we^ are .igoing to play a rol^e because organized labor is here and 
we're going to stay here. 

DISCUSSION: 



, MR. POULARD; I would hope also that the researchers are able to^ 
formulate some questions that would ^provide some insights regarding 
the factor that many t>f us feel is most important in youth prograips — 
the personnel directly involved in program operations^ NobleJ and . 
high-sounding goals of these pro-ams notwithstanding, it ^mes 
down to the selection of people who provide counselling and training 
and support services 1 

f remembes^ one case in Chicago where a grouj^ was asked to ,place 
only 20 percent of the graduates of i*ts very complex tpaiyiing program. 
With unemployment^ rampant jr a particular lady ended Up placing 8^%. 
She did so well that the prime sponsor apprcz^ached her and begged ' 
her on behalf of^he city to accept ai^ther contract to do placement ^ 
for other agenci^ doing the training'tor the. city. It's not because 
it was^a community-based organization alone, it was because those 
people were committed, were trained, and,.we2;e skilled. It was because 
the director of the, organization did not afesume that;jus4: anybody • 
'was qualified t<^ do it. They were carefuf in whom they hired. 

Again, I would suggest that the research*, if possible,, formulate 
some questp^ns to get some data regarding how critically important 
personnel are> in these* programs . It's not just the legislatipn, 
it's not just the volume of dollars, it's not just the program 
design. Success primarily is det^^lned by v5!feo's running it, ^ 
I think sometime^, ,at least at t^e level Qf reaulation or field 6 
memoranda/ credentials or requirement^- or qualifications for personnel 
should addi/efesed. . , v- * " ' » ' . / 



TAGGART: This legislation ' is qy'ite different from tihe rest of 



CETA in terms of labor uni6n and community-based organisations' 
involvement. YEDPA requires front 7end involvement" of labor unions 
and special consideration of community-based organizations. Tl^ ^ 
teason^tha% special consideration was given^ the reason labor' jjaon 
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front-end involvement vas put/ in the legi3lation, was because that 
was believed these alliances v/ould improve youth> programs • 

It is our job to assess whether these assumptions are true, y y 
The prociess evaluations are structtired to asaess whether community- 
based organizations do J>4tter than established institutions. In 
aa"dition to^TTat, we h^ye earlier discussed the HUD CDC project/ 
in which we ha^ comparisons between the CBOs*and other performers. 
We have^found that, 'indeed, community-based organizations haye had ^ 
good ;Leverage\ That is the finding, and 'it will'b'e researched, 
further. The school- to-work transition demonstration which has not 
been discussed has a group of CBO * s in varying circumstances bffering 
well defined, well structured, and sensitive servipds with impacts 
measured by post-tests. The Enjiployment Service^ ds -doirtg, t^i^ 
same thing, so that we ca^ test comparative effectiveness. . . ^* 

f Next year we're funding CSA to work with Farmers Home Adminis- 
tration to test the effectiveness of rural CBOs as oppose/3 to the 
balance-of -state delivery institutions. 

We're ^^oing the same things with 'labor union involvement and 
whkt we as researdhers have to'address is whether special considerations 
and labor union involvement have measurable benefits, 
t • 

MR. OSTROWER: I can relate somethings that^ we saw jn our studies. 
We' Saw that CBO's, of course, have greater participation under YEDPA 
than they did under the Title I program. EfSt we also saw that 'CBO*s' 
were more successful and more effective when -they dealt with things • 
like motivational training and job preparedness than when they weire 
l^eginning to deal "With things like t^raining and pther types of things 

them compete against th.e traditional agencies. Much 9f / 
th1?5 may have been learning curve problems I ^ Those groups that'are 
getting money for ^e first time and <^ing into cons.tructioh are 
having more start-up problems ' than those that^ have been at it for 
awhile. But what I think we're noticing is that* they *-re developing 
similar structures. * 



Another poijYt is that wlii er^ th e community^ based organizations 
have^a geograohic focus, they^have a^fconsfcituency^ and' they basically 
work with' thdife in that constituency. In other </ords, the^^^ are more^ 
likely to target; * \ . • ^ . . ' 

f4R., POULARD: I'm beginning to^ f eel that I've dpne CBO.' s a . great ' 
disservice. The last thing I meant to do was to make cdlhments which 
would pit CBO's against other Institutions* That is not 'a% all my 
point. Wy pdint is simply* that they've had ^to fight and force„ their 
way in.. Let all the oth^ grandma and--grrmd-pa--t>-rgaTrizations that 
ha\*^ been around sinqe Day One and that are still effective remain. 
If they can tirain better, God bless them, let them train. The 
only poinl!^ I was seeling to make is that there ought to be, a 
comprehensive' mix, the utilization of all program components ar 
institutions as v/ell. If CBO's are effective, th^n use them.- If^ 
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they can only do outreach, let them do outreach. If they can also 
train, let them train. At the level of pi^ception, of policy, of 
legislation, regulation, of field memoranda let there be some 
provision which ricognizes t*hem as an"^ entity, as other traditional 
entities .are recognized as entities, so that they are^ more easily 
involved in the process, rather than outsiders who have to always 
fight their way in. 'It's not one or the other; it's a., partnership. 

MR. OSTROWER: ^W^'re in thffe process of doing the comparison studies 
right now to determine whether CBO's do a better job of targeting. 
. It depends on whether the community-based organizations have th^ir 
own territories. In cities like S^n Antonio, * where the groups are 
funded through the prime sponsor and are "expected to have a larger 
territory, there's less targeting. - , ^ 

yiR. LEVIN ; When you look at tjie youth probjLem, one of the reasons 
we-a^re in it to a significant "degree is' because Of the sclopols and 
their inability or unwillingness to rfo a, significant ti;aining job 
with kids. There are ideas about alternatives! One that I'm 
thinking about is school vouchers. This "market" approach might 
make more sense. ^ , ' 

MR . TAGG&KT : V^e ' ve made a major commitment to' testing educational 
vouchers . We've completed implem.ent^atix)n studies, and i't's^ 
complex issue. We have seven million dollars next yea^ set aside 
to provide vouchers for participants in selected' work Experience 
programs. . \ ^ 

We are, in f act testing airT:he alternatives^ to schools we 
can, think of. We've got the Career Intern ^Pr^gram , 
alternative. The Job Corps is the largest alternative education 
program irr the country*. We've got educational vouchers, which is . 
another alternative. We've got the. CETA "system .creating all kinds 
of alternatives. We've got schools <;rea*ting alternatives under the 
t Exemplar^ In-School* Grant Program. The question is, which works 
V be3t and for whom? Some will do better with' some types of kids 

than other types of kids. We have to iji^arat^e all t^hat research 
•to assess comparative effectiveness, ^-ou'ye got to have all thfe^ 
angles on things, to assess them.^' That's why the structural - ; 
foundations- in the knowledge development plans are-^so important^ 
and that's v/hy everybody has got to know what- everybt>dy else 'is 
doing . ' ^ ^ . ^ ^ ^ 

MS. GUERON: . I'd jusb like to add that Entitlement's the most 
targeted of all programs; it is open only to. people on poverty 
levels or if the family is on welfare. There are some .difficulties 
in income co^nditioning the program, but I^^think there is going to 
be quite a good job of enforcing the eligibility criteria. To that 
extent, , it may be more tar^^ted than'^may be the other programs, 

MR, TAGGART; The most important fact is that there are kids out 
, there that want to work and are coming out of th« woodv/brk, if 
v;e off^r them even minimum wage jobs, much less jobs^that^ pay 
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higher. -One thing Ve' re going to prove by /the raw numbe^rs is 
somethi^ng realistic s^bput , the universe of need. 

MR. HAUSMAN: One thin^ that amazes me is how many *agencies--and 
now yours, too — are in! the business of determining eligibility by 
income and other things. One time or anotlter, we're going to fund 
a research prefect to pee hov; much n\oney we are spending per family 

ity for several different programs, whose 
don't vary too much. 



to look at its eligibi 
eligibility standards 



f 



MS. GUERON; The difference is that you can bring in yo^\r welfare 
stub and that's the ticJcet to eligibility for^ Entitlement • 

f4S . V7ILLS; . I'd, like tp perhaps speak the unspeakable on this question 
as to whether we're seirving the hard core. I'm not sure that it can' 
be done in this next yfear or two years, but I think jwe're going to - 
have to look eventually* to reasons why the dropout or the younger . 
individuals simply do npt participate in soitie of these programs. 

. ^ My conversat-ibns7 interestingly enough, with a lot of CBO's 
indicates that they aire seeing a tendency for what they're 
calling the ""CETA husytle", in the local areas-, where the kid comes 
in and he drop^ out and knows how long he needs to b^ unemployed, 
and/or is noticing to take the job,? because frankly he can make 
more money in other economies of our society. } 

I tHink you're ^joing to have to address that. .One can get 
fancy and talk aboutTreservation wages and all theseuother things, 
but I don't think it?s really necessary, I think that— in the next 
year 'or two y6ars,\yQu ' re >going to have to think a±)6ut those you 
^are really calling ItKe hard core. 3^ thinJ^^we ' re " going- to be forced 
*to begin to address Tnose kinds of questions, I think it raises 
ajjtestions from DOL perspective, ^as to wh^re are the limits of ^ . 
government intervention in terms of employment and training programs, 
and I think^it needs to"be approached from that v/ay. ' 

\ One other comment in terms of disseniination . I^heard a lot , 
of words about ^replication yesterday. And I v/ould suggest to the 
representatives in the Department^ of Labor that perhaps one of the 
thifi§*s that Ve need to think through a little bit more is to think,' 
througl^ a strategy for r^al dissemination of information. The 
.project operator or the user^ out ther^ in Mississippi may 
or may not havi^ access to or interest in documents. Everyone in 
th6 wprld ca^r^ for a better training and technical assistance 
from the Department of Labor, but perhaps one of the things that * 
we can do^^is with utilization and the ^regional training 'grants 
and a whole systeiB^ • strategy is go beyond that. 

And I would think also that the approach of developing a cohesive 
knowledge development plan, itself, needs to be replicated in other 
parts -pf ETA. 
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Prepared by: ^ Manpower Demonstration 
R5search Corporation 



Bii? paper summairizes research activities on the Youth Incentive Entitlement 
Pilot Projects tihat will be conducted by the Manpower Demonstration rtfesearch Corporation 
(MDRC), and Abt Associates, Inc. (Abt)^. TMDRC is responsible for the development and 
oversight of all research activities on Entitlement. At MDRC, Joan Leiman is the 
research manager, and Joseph Ball axxd William Dia^ are responsible for the design 
and conduct of the implementation analysis. At Abt, Ernst Stromsdorfer/ the 
principal investigator, and Robert Jarrett, III, the project manager, are responsible 
for the intact analysis. » 



SUMMARY OF RESEARCH ACTIVITIES ON THE 
YOUTH INCENTIVE ENTITLEMENT PILOT PROJECT 



4 < 

The Entitlement program (Title* Subpart J of the YEDPA), is designed 



aa ^ demonstration — 2Cn experiment—to (1) test whether school-year (p€u:t-time) 

^ / 
and summer (full-*time> jobs can be guaranteed for 16-19 year old/ economically . 

disadvantaged youth residing in a designated £u:ea who ar^ in secondaory school 

or who are willirfg to return tp school or enroll in a cotirse leading to 

h cert^ica^te of high school equival^cy; and (2) test whether such jobs 

will be an incentive to increase high school ^retention, return, and completion, ^ 

and 'longer ter^v employment and earnings of program participants. 

^ %^ Within designated affeas during the school year, otherwise unavailable 

* ^ \ ' \ 

ft 

part-time c.i^oyment or. a^ combination of part-time emploi^rtent and training 
' is guaranteed to tho^e economically disadvantaged youth" between the ages 
' of 16 to 19 inclusive^|?iio cure in secondary school or a program leadinCf to 

* ' a certificate of high., school^ equivalency. During the summer, full-time 



employment or a commnation of employment and training is provided to all 
eligible youth. 

Because of the high cost of EntitlementT^ly a few^ full 
scalei programs could be launched that would test the saturation of an 
erlti:pe central city or multi-county curea. In order to test ^ number , of 

/ ■ ' / 

secondary experiments and innovations proposed by the legislation, the 
Department ojE Labor specified a "two-tier" approach. Seven large-scale, 
or Tier I, projects test whether CETA prime sponsors cjm feasibly 
itt4>lement programs that will encompass all or 'a large segment of thexr 



service area 



. Tipp/I^^demonstrations a^e expected to ^^\|ide ^jobs^f or 



between. 3,500 and 10,000 youth. Somewhat Smaller Tier II projects, each 
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covering the area served by a psorticular school or small school district, 
test a greater variety of programmatic approaches • Each will enroll ^ 
no more than 1/000 youth and the. majority will enroll less than 500,^arti~ 
cipants • ^ 

To meet the legislative mandate which specified diversity and required 
estimating the probable costs of a nation-wide entitlement, sites were 
selected to represent a wide economic and geographic rangfe — both large and 
small cities, high-density urban areas and sparsely populated rural regions < 
and ar^as of varying unemployment rates, school dropout rates, and /acial/ 
ethnic compos ition. 

In section 329 of the^ Youth Act, the Congress has directed the>Secre- 
tary to report findings on the efficacy of the entitlement projects with 

< a 

respect to: 

"(1) the number of youths enrolled at the time of the report; • 
"(2) th^^Mt of providing employment opportunities to such youths; 
"(3) the degree to which such employment opportunities have caused 

out-of-school youths to return to school or others to remain 

in school; 

" (4) the number of youths provided employment in relation to the 

total which might have been eligible; 
" (5) the kinds of jobs provided such youths and a description of 

the employers - public and private - providing such emp^^oy- 

ment ; 

" (6) the degree to which on-the-job or apprenticeship training has 

been offered as pzort of the employment; 
" (7) the estimated tx>s\ di such a program if it were to be extended 

to all areaa^; 
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'*(8) the effect su<9h employment opportunities have had on reducing 
youth unenployment in the areas of the prime sponsors operating 

4 

a project; and 

"(9) the in^>act of job opportunities provided under the project 
^ on oti^er job opportunities for youth in the area." 

Additional research concerns can ^e inferred from the Statement of 
Purpose to the demonstration projects Title of the .Act (section 321) : 
• (10) "It is explicitly not the purpose of this part to provide 

makework opportunities for unemployed youth; instead, it is 

the purpose to provide youth... with opportunities to learn and . 

rthat will lead to meetningful' employment opportunities ' 
alter they have completed the program." 
Finally, the Ac^ specifies congressional interest in understanding 
the importance of the service delivery system for implementing entitlement, 
with a stateioent that the purpose of the demonstration projects shall be: 
(11) "to test the relative efficacy of different ways of dealing , 
with these problems in local contexts. (section 321), and by 
• providing for tests of 'a variety of administrative mechanisms 

to facilitate the employment of youths un4er ary entitlenjent 
arrangement...*" (section 327 (b))." 
The Congressional mandate is reflected in the Office of Youth Program* 
Knowledge Development Plan which addresses the requirement that the demon- 
strations test the e/ficacy of Entitlement under differing socio-economic, 
regional and governmental 'Circumstcoices (p. 10a); assess, the feasibility 
of Entitlement for entire prime sponsor jurisdictions and estimate the 
probable costs of such proqramy; describe the scopo and nature of jobs 
which *woMld be created; and assess the program's impacts on labor markets 



and school completion and retention rates (p. 11). In addition, the Plan 
anticipates that the research effort for Tier I projects will also examine 
the long-term impact of the program on participating youths in terms of 

A. 

their post-progr^ labor market and other behavior. Finally, the Plan ^ 
notes that the research is to be concentrated in the Tier I projects, 
with a qualitative or process assessment of the smaller Tier II projects 
which would examine the range of new and improved approaches, methods, 
and techniques to be explored ahd demonstrated at thesp sites and would ^ 
also generate knowledge to complement the Tier I analysis (pp. 12-13). 

Three major issues will be analyzed to meet the congressionally mcindated 
research findings and the Knowledge Development Plan-: 

a. The impact of the program (on participants, eligible youth, 
and other youth) ; 

b. The costs of the program; and 

c. The operation and coordination or implementation of program 
service delivery mechanisms. 

The Department of Labor has contracted with the Manpower Demonstration 
Research Cojqjoration (MDRC) to conduct these impact, cost, and im^^lementation 
analyses, and to coordinate the implementation of the Entitlement demon- 

/.3tration under the supervision, management and direction of the Office of 
Youth Progifams. MDRC has, in turn, subcpntracted with two firms respec- 
tively for the conduct and analyses of program impact surveys with eligible 

. youth; Mathematica Policy Research of Princeton, NJ and ABT Associates 
of Boston, MA* Implementation and cost analyses will be cont^ucted by MDRC 
research staff. Each of the analyses and the congressionally moated issues 
and questions which they will address are discussed below. 



THE RESEARCH QUESTIONS • 

IMPACT. ANALYSIS / 

'Several of the mandated resea^^ch eureas concern the impact 'of the 
Entitlement demonstration upon youths in entitlement areas and upon the 
labor meurkets which they comprise « . With reference to the legislative 
questions enumerated cibove, the impact emalysis will address the following 
issues and hypotheses: 

♦ 

A. Participation rate of eligible youth (1, 4) f 

V a. What proportion of eligible youth enroll in the demonstration? 

c 

b« What are the socio-economic-<3emographic characteristics of 

enrollees?, ' ^ 
c. How do peurticipants compare to n^n-participants? 
« d* What program design cind environmental factors explain partici- ' 
pation? 



B. Short-term educational attainment eind school performeince (3) 

• J" 

a. What is the impact of -^Intitlement on the performsmce of ^ 
students already enrolled in school and on their retention 
in school? , , 

b« What is the impact of Entitlement on the return to high school 

or GEIX programs by former high school dropouts, cuid their 

4 performance in such programs? * ^ 

€• iShort-term- impacts on employment ^ unemployment, and Icibor-force • 
peurticipation of poor youth (8) 

a* What is the impact of Entitlement on the employment and unemploy 
ment rates of students?" , ' 

What is the impact of Entitlement oh the employment and unemploy 
ment rates of former dropouts? 



r 



D# Longer-term impacts on earnings ^ post^secondary education and 
expectations (10) ' 

a. What is the impact of Entitlement on the post-high sci^ool 
earnings of students and former dropoutS? . " 

b. What is the impact of Entitlement on the college entrance 
rates of stjidents and former dropouts? 

C 

c. What is the impact of Entitlement on peirticipant expectations? 

E» Effects on the labor market for non-poor youth (8, 9) 
< r 

COST ANALYSIS 

'^The cost cuialysis, to be conducted by MDRC, will address issues (2) 
and (7) in the legislative mandate. Data will be drawn from the Entitlement 
fiscal reporting and information systems that have been designed and are 
being managed by MDRCi Data from these systems will be used to determine ^ 
total program costs, costs per peirticipant, and 'costs per peirticipant- 
yeax for different subgroups of youth. These unit cost Measures, 
combined with the sample survey estimates of program participation for the 
different groups of youth, will be used to estimate the cost -of extending . 
Entitlement to all areas of the country. 



IMPLEMENTATION ANALYSIS 



The purpose of the implementation analysis is to understand and explain 
the programmatic development of the Entitlement demonstration at the sites 
and draw lessons concerning the demonstration's operational feasibility, both 
generally and under varying site circumst^ces. Although there is a basic 
program design for Entitlement under the management of CETA prime sponsors, 
there is consideraible variation across sites in the administrative arrange- 



»ents Established for operating the program. The implemlentation" analysis 

is examining the influence of these varying institutional arrangements, 

f 

processes, and decisions upon the program's development, as well as local, 
political, social, and economic factors that may also'' affect the way in which 
Entitlement operates. 

The general research on the implementation of Entitlement will cover 

. / 

a number of key cireas which cire discussed below (for each cirea, the corres- 

' ■ W 

|>onding question in the legislation is indicated parenthetically) 

A. Program Description (1, 5, 6 ). 

This will be straightforwcird description of program content and 
operations: ^. , f 

How many* youth were enrolled? 
What are the chciracteris.tics of enrollees? 
What kind of work did they do? 
For what kinds of employers? 
What kind of schools did t^ey attend? 
What roles did peirticipating agencies perform? 
In addition to providing narrative descriptions of what happens in the 
Entitlement demonstration, .this will be .a principal data base for other 
areas of implementation analysis discussed below. 

B. Faptors Affecting Participation (4) 

Participation at each -site will depend oh the specific form in which 
the xlesign is implemented as well as underlying lopal conditions and youth 
preferences. Jhis component of the research will describe those , aspects 
of program implementation considered most relevant to the youth's decision 
to participate in Entitlement and, as such, will complement the impact 
analysis in e3q)laining the participation rates at the sites. The data collec 
tion will focus on recruitment and enrollment procedures, the nature of jobs 
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and the worksite experience, the' educational alternatives available and 
the ixDplementation of eligibility monitoring and grievance procedures.' 
It is assumed, for examj^le, that participation will be higher where there 
is an aggressive outreach effort and where, for another, example , youths 
are offered a variety of ' educational choices, 
* C* Schbol-Pri!me Sponsor Linkages Under Entitlement (11) 

~7 T '• 

Tnis study will document and analyze the roles played by the two 
primary agents in ^J^pl^enting the two "halves" of the Entitlement offer. 
Although school systems and prime sponsors have generally established a 
number of working relationships over the yeeirs, Entitlement will provide 
these two institutions witln the opportunity cuid necessity to coordinate 
work experi^ce and school services to a greater extent than has usually 
• been the case with in-school work programs. In some "bases, the relationship 
will be ^an entirely new one. Since the schools, and prime sponsors have 
— overlapping but different senses erf mission with respect to youth and employ- 
ment. Entitlement offers ,the chance to explore the conditions^ under which 
such coordination works effectively and to asse^ss the lessons this pairtner- 
ship provides for carrying out a guaramteed youth job program elsewhere. 

D. The Adaptability Of The. CETA System to Implementing\ Large Job 
Gucurantee Demonstration (11) 

There will b^ an effort to^ docvnaent the problems encountered in^ prime 

si>onsors* implementing a job guaramtee concurrent with ongoing (and possibly 

con^eting) program responsibilities. Problems may arise, for example, from ^ 

an assumption of resource, scarcity on^^e part of the prime sponsor derived 

from past experience with programs whose limited resouj^'ces *have required ^ 

, careful selectipn of sub-populations ("significant segments") to receive 

program services.. .Such an assumption in Entitlement, which ^s designed to 

> provide all eligibles with jobs 'upon request, may lead to overly cautio'us 
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pr^ram implementation • Other problem^ « may arise from the imposition of 
a demonstration pr^ram on the decentralized CETA system wher^ program 

> 

discretion for prime sponsors will be more limited them with other CETA- 
programs, 

E. Tier II Inndvati<^s . (5/6) , > 0 

f Most of the implementation analysis will be concentrated on the larger 

Tier^ I sites. However\ there will be a systematic documentation of the • 

program innovations introduced at the Tier II sites. Reporting on jt:he • 

special^innovations will concentrate upon comparison l?f experience with the 

saii^ program feature across sites, and upon lessons concerning the replica- 

tion of those innovations that appear most effective. Eff ectivehess will ^ 

be considered not only in the context of a job guarantee xander, these special 

» 

arrangements but also in the more general context of exemplary program 
services for disadvantaged youth. 

The general implementation, research will be ^J)plemtented by special 
Implementation ^studies in thr^e. important areas. «s / * 

F. ij^ivate Sector Work Experience (5,6) 

Given the unique wage subsidy provisions of the Entitl^ent demonstra- 

■ / 

tion which allow a wage subsidy of up to 100% to private septor firms that 
provide program worksites. Entitlement sponsors may seek to develop a stronger 
private sector involvement than has typically 'been the case with ^youth 
work experience programs. A special study will examine the role of the 
privates sector in the 'demonstration, the subsidy arrangements that were ^ 
utilized,-* other arrangements to make participation in the Entitlement 
program more' attractive to the private sector, and the effects of these- 
arrangements oh the volume of private sector ^job creation. ^ 
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G. Rural Programs ^ 

** • . 

The 'demonstration includes programs in rural areas of Mississippi, / 

% « . • ' / 

Steuben County (NY) and Monterey (California). In studies of job'cr^tion 
and enrollment, there will be p^ticiilar emph^is on the difficulties in 
creatii^g jobs in rural areas and the arrangements that developed between 
the' employers and the schools to deal with transportation and other coor^i- 
nation problems . Of partiicular interest is how diff erent iti scope and , 
type these prdblems are relation to thoge that may arise m urban, sites. 

H. The Quality of Work (10) ^ 

As part of its monitoring re'sponsibilities, MDRC will survey a sample 
of program worksites to develop a profile of the quality of the work 
experience at the different, programs. The importance of the Entitlement 
worksites lies in their ability to provide an exemplary work expej?lence -that 
develops and reinforces the positive aspects of work for youth. Therefore, 
we havel tentatively identifiecf five factors on which a sample of at least 
50 worksites for each .Tier I sil^e^will .be assessed. The^e factors / derived 
and adapted from the literature on job quality of earlier work experience „ 
programs, are: ^ 



a. . The content of the work provided; 

b. Thev^or^toization and management of^the worksite; - ^ 

c. The level and nature ^of superv^^sion provided;' . • ^ 

d. .YoutH^per.ceptions of the. value of the assignment; and ^ 

e. The v4lue of the work to the sponsoring*" agency or firm. ^ 

t * — 

The discussion that follows addresses in some detail the research » 
design and analysis strategies,,ahd imitations, for responding to the 

above issues and questions. These will be discussed, first for the impact 
analysis followed by. a discussion concerningj the implementation analysis. 



I^ESIGK ISSUl^S 



IMPACT ANALYSIS „^ ' 

r , ' ' 

* Research Design and Analysis Strategies . * . 

The key issues and hypotheses identified above for the impact analysis and 
coherent time-sequencing of program treatments and' effects present the ixnique 
opportunity to trace the entire process of pr^ram enrollment and program * - 
operation through the receipt of treatments ultimately , to* obsexrvfe^ both" 
the intermediate impacts and final impacts of the program, This process 
and sequence of events comprise the followiig general analytic models of 



c 




behavior: 

1, oA model of pre-progreum behavior, ' 
The purpose of this model is to exploit the baseline survey 
data to describe the work/schooling experiences of the sub- 
jects prior to Sui^mer 1978. ParticuJ^ attention will be 

' given Ax> part-time versus full-time work e>5)erience: s^pmmer 

jo}J^versus jobs held during the school year: wages .received, 

^ ' * <* , . 

the nature of reservation wages and their effects on ^abor 

i 

market behavior; and the nature ^d extent of^ time spent in 
school,^ ' , . 

2. A model of participation. 
The piirpose of this model is to: 



a. estimate initial demand for ^he program; 

b. model continued enrollment, late enrollment, *and^' 
attri tion ; and * * 

c. -provide parameter estimates whereby program pa^icipation 

can be used to generate an instj^^ntal variable to serv^^. 
as one control for self-selection bias. 
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^ is 



/expected that participation will be examined ' 



separately for Spring 1978; Summer 1978; Fall 197S, 

ana tlvereafter* Of particular interest will be the 

.propensity pf Summer enrollees to honor their commit- 

ments to attend school during the Fall. ^ 
A model of intermediate time allocation program impacts. 
This model examines labor^supply, time spent on school\^rk, 
and time*devoted to other activities dui-ing the secondary 
school yecirs. Summer activities and school-year activities 
will be considered separately. 

Ii^or supply. Daily, weekly or total program enrollment 

hours spent in the labpr market; • 
b« Schooling (educational production functions) : 

bi length of additional weeks or months spent in 
secondary school 

b2 the probabilityVof secondary school graduation 
— b3 change in grade level completed 

b4 change in grade point average 

b5 change in cla3^ standing 

bg change in school attendance: 
i) change in times tardy; 

ii) ci^nge 'dn number of days absent, given the required 

f . 1 ' r . 

. ^ a%t:en)^^ce ^e riod ; 

III) change in weekly hours spent in school building^ and 
c» Leisure and other activities* ^\ 



— - These are intennediate program outputs> which become final 
program inputs in models which estimate posts econdciry ecluca- 
tion and labor market impacts. It is important to examine 
these, intermediate program impacts separately during the 
summer and the school year. Close attention will be paid 
to the qucmtity cmd mix of school courses or other types 
of training or education that occurs. 
A. * A postseconB^ary labor supply euid employment effects model. 
The purpose of this model is to measure the size and statis- 
tical significcm'ce of the intermediate program impacts on 
final labor^mcurket impacts. Thus, the secondary schooling 
or other training ,cmd education effects which were interme- 
diate outputs above become final program inputs in the model 
of labor supply cmd leQaor market effects. 
5» A model of the demand for postsecondary education.' 

The purpose of this model is to complete the estimation of 
desired program final outputs or impacts. — Againr- -the~seeon— 
dary scho&Iing or other training or educational intermediate 
outputs become final program inputs in this model. 
6. It should be noted that the labor supply modei and the 

postseconc^ary schooling demand mOdel caih be combined into 
a simultcmeous equetion system whereby hours spent in the 
labor market and hours si)Ont in postuecondary schooling 
Ccm be joiatly estimated. 
A variety of statistical and econometric approaches will be usqd to 
perform the above analysis.* It is important to note that while relatively 
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complex behavior will be^odelled, the salient features of^ analytical fiifdings 
will be systeiq^tically displayegl and discussed with appropriate cross- 
tabulations developed either from a standard analysis of variaxice approach 
or through more complex regression procedures* Complex econometric models 
will^ot' encumber the main body of any report. 

The data required for the impact analysis will come from several 
sources. First, a household survey was conducted in four Entitlement 
sites amTTour control si teg.* By means of screening interviews administered 
to an estimated 120,000 households, a sample of about 8,000 eligible youth 

0 

has been selected and interviewed, 2/3 from the pilot sites arid 1/3 from 

" ■ - / ^ ^ 

the dbntrol sites. 'The first j (baseline) interview was administered in 
early 1978 to the eligible youth and their parents (if they* live together) . 
the second interview will be administered in early 1979 to youth and their 
parents; the third interview will be administered only to youths in early 
1980. ^d the fourth and final interview of youths will b^ administered 
in 1981.^ The exact mix of youths among Seniors, juniors, sophomores, and 
freshmen has not yet been decided for the second, third a^;id fourth interviews, 
A second data sourcejwill be the recojrds of the high schools and GED 



programs in the four. Entitlement arid four control sites. Finally, data will^ 
be collected through the demonstration's information "system d^gned and 
operated by MDRC. ^ , 



Research Limitataons ^ 



Perhaps the major technical^ problem in this fitudy is that of proper 
estimation of the program participation function and the use of this function 
to-help overcome the problem of self selection bias. 



'*The four Entitlement sites are Cincinnati, Denver, Mississippi (selec- 
ted counties) and Baltimore; the four control sites are Louisville, Phoenix, 
Mississippi (non-Entitlement counties) and Cleveland. 
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Participation brea)cl3 down into two components. One must first predict 
tlie B^bability that a person will join the progrcun. Once having joined, one 
must predict the extent that one participates. While it is possible to use_^ 
the immediate benefits of the program, such as earnings on the* subsidized 
job, as predictors of the extent of partic^ation, it is not possible to use 
these benefits. to predict whether one enrolls or not. Rather, independent 
or exogenous variables which are not diregtiily related to short term program 
benefits must be used to perform this task. Here we come to the problem of 
selr-selection bias. ^ ^ 

S6lf-^e£ection bias is essentially an omitted variable problem.^ In ^ 
other words, crucial variables needed to explain the likelihood of participating 
and succeeding in the program are missing. To give one example, if "ambition" 
is omitted from the analysis to predict ultimate educational attainment, and 
'if, other things equal, more ambitious youth select . themselv^^into the group 
of program participants, tjien the estimate of net program effects will be 
bi&sed upwards. That is, the self -selection effects of ambition are 
a^ributed to t^ effect of program services. 

In the Entitlement study, this problem is substantially corrected through 
the use of a household survey to identify eligible persons in both the prcfgram 
site and a control site prior to program start-up. Each site contains 
eligibles, but, as figure 1 shows, in the program site, there are both partici- 
pant and nonparticiparft eligibles. Under the assumption that the program 
has^no ef f ect ^upon nonpar ticipant eligibles, we can attribute any difference 
which is observed between^ t^j? group and the controls to the fact that tmis 
group consists of the individuals who did not select themselves into^ the 
program. When the participant eligibles are joined with the controls to form 
a data ^t for analysis and analysis is employed in which a dummy variable 
identifies "experimentals" versus "controls," the measure estimated for this 
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dummy veuriable is the stan of the program effect and a selection effect. 
If we knew the magnitude oh the selection effect, we could subtract it out 
and thereby find the true program effect* But we can measure the> ^selection 
/ effect by creating a different pooled sample, one containing nonparticipaht 
eligibies and eligible controls # and estimating program effects for them, 
with the ssj^ral groups identified by a dummy variable. Then, the measure 
. associated .with the dummy variable in this calculation is wholly attributable 
to selection. 

♦ 

Figure 1 

Program Control Site 



Non- 


Participant 


Participant 


Eligibies 


Eligibies 





Eligible 
Controls 



Dummy Variable Coefficient 

« Treatment Effect + Selection Effect 



Piammy Variable Coefficient « Selection Effect 
The result is an unbiased estimate of program impact. 



Another limitation to be noted is that rigorous measurement of the impa<^t 
of Entitl€[ment job opportunities on other job opportunities for youth, including 
"non-poor youth (question 9) f would require that a random sample of all youth 
in the selected areas be surveyed. It was determined that the costs of this 
sample would be prohibitive, more than doubling the sample size and cost. 
The sample size was determined, therefore, by the need to answer with sufficient 
confidence the questions of short and. long-term impacts only for poor youth. 
Other data sources will have tq be utilized to develop an approximate estimate ^- 
of Entitlement's impact upon non-poor youth employment, and these estimates. 

r 

will not permit the same level of statistical confidence which would be 
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*See page 3, above. 



possible, through surveys « 

IMPLEMENTATION ANALYSIS 

Research Design and Analysis Stfategles ' ^ 

Utnplementatlon analysis addresses the behavior of key "actors" 
(institutions, agencies, flrfas) who participate In the delivery of Entitlement 
services to partlclpfimts^ and the effects of ojKer'^cal circumstances on the " 
progi?am. In order to describe and account for the behavior of service 
dellverors, it is necessaary to understand major features of their structures, 
mlQsions and organizational stedces in Entitlement,' previous program operat;ing 

3ience, and previous working relationships with each other. The way in 
these factors come together for an organization will help to account 
he way in which the organization carries out its role in Entitlement 
and the extent to which the organization's members perceive an incentive to 
cooperate with the other actors delivering Entitlement services. 

*A dynamic program implementatibn process in unique l^ocal settings, 
particularly when this process involves many different actors, with different 
histories and administrative arrangements in the several Entitlement communities, 
does not lend itself to highly quanti^tative measurement.^ It is important to 
derive an understanding of Entitlement processes through the collection of 
data about program decision^ and actions, alc^g the range of qualitative 
vai^iables mentioned above, from the viewpoint of all participating actors, and 
where possible, by "triangulating" the observations of morer than one outside 
observer. Building such an underst^kiing is a holistic process, and multiple 
actor viewpoints do not constitute redundancy. The data sources which are 
being utilized in the implementation analysis^ reflect this conception. 

The sources of qualitative program data include outstationed field 
monitors assigned to the Tier 1 sites, MDRC central field staff who periodically 
visit the sites as part of thei*^ oversight function, and cohsultants to MDRC. 
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All of these have resppnsiblity for dociimenting and^assessing vario^^^s aspects 
o£ program implementation through periodic, structuj^ed research reports. The 
out stationed field monitors responsible for desCiribing and assessing basic 
program activities such as recruitment and enroLlmentr schooling, job creation, 
and others in a series of structured research reports. In addition they will 
carry out the field research for the quality of work study (see below). 

Central MDRC field staff, and a group of consultants under subcontract to 

0 

MDRC ta monitor five Tier II sites, contribute, to the research through sharing 
of their field visit reports and through periodic debriefing with the research 
staff. In addition, they contribute through structured research reports on 
the management and coordination of the demonstration by prime sponsors and 
their managing agents. 

To enable a more thorough analytical assessment of the development of ' 
Entitlcmei^ at the Tier I sites, the implementation analysis design also calls 
for a series of interviews with the prime sponsor and other program agents 
during the course of the demonstration. These interviews, carried out by MDRC 
consultants during a week's field visit, are structured to address background 
factors at the sites that may explain the program's development and content. 
Such fact6rs incljide^ local political conditions, the quality of the relation- 
ship among the various program agents, their perceptions of the program and 
of its problems \nd successes, and the rationales behind administrative 
decisions and oj^anizational arrangements. 

Finally, quantitative data on progr^ participants and content is being 
provided bj/ MDRC's Entitlement" Information System on a monthly and quarterly 
basis with special reports produced as needed. This system, which in addition 
to informing the implementation analysis serves other MDRC units and rep^ing 
retirements, produces data on the number of enrollees, their demographic 
and employment backgrounds, their source of referral, school enrollment data 



by typ^N^ school and type of curriculum, participant activity in terms of 
initial 30b or training assignment, termination and re-entry where applicable, 
characteristics of the worksite or training sponsors,' monthly levels of 
youth participation by types of jobs,iRfcraining, and wage rates, 
and total wages paiid to individual, participants^ * 
Research Limitations ^ 




One research limitation that 'should be noted is that \he quality-of work 
study will not be a highly rigorous and quantitative evaluation of the work 
^perience provided to participati/ig youths • ' Due to the part-time nature 
of the majority of the work provided during Entitlement, the fact that it 
is a youth program ^designed to primarily provide a good work experience 
rather than to impart specif ic vocational skills, and other cost and data 
limitations; a qualitative and. extensive worksite «issessment strategy is 
pleinn^^^s^ It is anticipated that from 350 to 500 Tier I worksites will be 
assessed by the Tier I monitors u^ng a structured assessment instrument. 

Another potential, limitation is the reliance of the Entitlement 
Information System oii data recorded And processed by local prime sponsor staff. 
During program start-up, there tiave been numerous and serious delays in the 
delivery of data by prime sponsors to MDRC's, central infoi<m^|lon system staff. 
While the problems have largely been alleviated and data from most of the sites 
are now flowing in a .satisfactory "way/ the system obviously still remains * 
vul^ierable to unforeseen difficulties. 
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TIMETABLE OF RESEARCH PRODUCTS 

■ . J" • . ■ ■ ' •" 

Tha Entitlement demonstration and research activities are still in 
an early stage; the demonstration began enrollment in late spring 1978, 
the baseline survey for the impact analysis was completed in the summer of 
1978, and a variety of interim reports and interviews on\^ start-up period 
have only recently, been completed^ The following research reports are^ 
tentatively scheduled for the dates indicated: / 

■ ■ ■ ^ ■ ^ ^- . 

Impact Analys^ Reports * ^ v 

* Baseline Report ^ January 1979 

• * , J 

Participation Study January 1980 

Report on In-program Impacts February 1980 

Report on Post-prigrapi Impacts January 1982 

■V ' ^ . 

In^lementatioR Analysis Reports , . 

. General Implementation Report January 1980 

The Quality of Work in Entitlement January 1980 

The Role of the Private Sector , ^ 
in Entitlement October 1979 

Entitlement ^n Rural 'Areas ' February 1980 ^ 

♦ 

In addition, MDRC is preparing a report on the early program experience 

»» 

as specified in the' YEDPA (seotion^329) . ^ 
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POLICY IMPLICATIONS 

The potential policy implications of the research obviously follow 
from the questions and issues articulated by Congress and the Office of 
Youth Programs as presented earlier in the paper and as explicated above, 
Among the major issues to be addressed are: ^ 

1. The operational meaning ,and feasibility' of operating a guareinteed 
"^5^s program lander varying local circumstances and administrative 

arrangements . 

2. The impact of such a program, when school-conditionedy on the 
school performance and labor market *behavior of participating 
youths both during their participation and beyond it* 

3. The costs of operating ,such a pjro^ram and the potential cost 
should it be extended. ^ u ' 

4» The ability of prime sponsors to create jobs in the private 
sector under wage«»subsidies ranging up to 100 percent. • 
5* The quality of work that can be provided in a large scale, rapidly 
' implemehted work experience program for youth. ^ > 
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I. > OVERALL PURPOSE OF THE DEMONSTRATION RESEARCH . * |^ 

As noted in the Knowledge Development Plan for YEDPA, "the underlying - 
objectives of this demonstration project areV: I ^ 

(a) to demonstrate and test the feasibility and ig^fectiveness 
ofla replipatinR methodology itself (i.e., how and under 
what conditions can the pro'granf model be adeqjliately re- 
plicated in various qommunities.) ' j ^ , 

(b) to develop and refine A "work evaluation" methodology which 

may eventually be incorporated into the f onft|[ila-f uncfed community 
Improverofent type efforts. 1 ^ ^ 

r - ' (c) to compare -the post-program experiences of youth partici- ^ 
pating in the replicated projects (which will mainly pro- 
vide job experience) with those for a compatafale group of 
young people of similar backgrounds who have participated 
in other manpower programs (which have emphasized classroom 
training, manpower and support services) ih the same loca- 
lities." (p. 17) • 

11^ SPECIFIC RESEARCH qUESTIONS, ISSUES. AND APPROACHES • 

/ * 

II. I' The Feasibility and Effectiveness of a Replicating Methodology 

Consistent with the Knowledge Development Plan, CPPV conducted "a review 
of both appropriate post programs and designs of projects financed through the 
formula allocation for"^ Community Improvement" (p. 17) and selected a model pta- 
gram, t)ae Etnergen^y liome Repair program (EHR) of Portland, Oregon. This Ao- 
grara utilized in-school^and out-of-school youth to comf)lete home repairs tor 
poor, elderlv, arid handicapped homeowners.. In addition to meeting basic YCCIP 
criteria (e.g., the target population and the production of tangible, long-last- 
ing corOTunicy improvements), the EHR program boasted several aspects that were 
especially effective: \ 

. Close union involvement : EHR^ youths work in small teams. Each 
team is supervised by a crew chief who is a journeyman member of 
. the local carpenter's union. This aspect not only appe^Jred con-^ 
ducive to good skills training, but. was cleemed'asxa positive as- 
set in subsequently motivating youths to seek entree into the build- 
ing trades" unions. 

/ . An effcctIve>ommuaitv agency UnkaRe system : EHR, in its planning 
mid operati9h, effectively involved a number of local agencies 
(schools, unions, courts^ and a-variety of municipal bodies) in 
order to: ^ ^ ^ , 

^ provide sound program management 

refer ample and interested youth from the target population 

^ . generate a s^table inventory of work projects in a timely 
manner ^ . 





• obtain building and licensing petmifes/apprpvals 

•N^rovide po3t--program employment, training and edui^lon , 

The successful transfer of the EHR model to diverge jurisdictions mu^T^e 
Jbased on a clear distinction between essential, characteristics of the model and 
Incidental features of particular program operations and particular ;1uri3dictlon3 o 
For examp^u .while direct union fnvdlvemerit via the designation of journeymen 
crew chlMpK^s deemed essentia, it was not^de&med essential that the one 
particular union need play the lead role In all replication efforts. This dis- 
tinction between the essential and the incidental is critical, because circum- 
stances vary from one locale to another. The process ^f replication must, there- 
fore, maintain the ^essential principles integral to program success and yet he 
flerxlble and sensitive to the needs and conditions of diverse communities. 

Based on an intense analysis, CPPV distilled^ nine (9) essential program 
elements which sites who participated in the demonstration are mandated to 
Incorporateo These elements are listed in Appendix Ao ' 

The ef ficeficy of the VICI replication process will be assessed primarily , 
by means of process evaluation which will yield a documentary analysis of 
issues such as: - - 

The^x.tent.^ to. which linkages required to implement the model are 
-^rc-existent in Ibcal'communities ^fid/or the processes involved 
in creating. thoSe linkages locally^ v ' • ' 

o The ^^politlcs o^f eplication" in local areas— ie e<o « the extent 

* * to which idltp^ncr^tic ^^ocal factors (quality of leadership, prior 
^ history of 1 inter-ins titufelotikl cooperation, local politics of 

^ CETA, etco) enhanceH)f imj^edeji&uccessfui repXicationo 

o The speed with which l^cal sponsors^ mount? programs, their ability 
•to attract^and retain the target' pppulat^j^ in tfie program, ^nd • 
thelt ability to perform valuable work -on^ timely basis. 

• • The extent and nature of the quali^:/''o^^ inter-ins tltutional }.i*nk- 
^ ages exhibited during program operation.. . * , ' 

, * ' ' ^ % - . . " ' 

o Assessment of ways in which ^ocal economijc coq^tions, political 
* climate, and city scale impact the program. ^ 

o Ways in which local preference^ inifiatiw^r conditions produce 
local inhovations and augmentatioij td the. model; 



. Evidence of "spin-offs** from succ^s^ful program implementation. 

To ficcomplish this procnss or documentary ri^arch, CPPV has retained the 
services of two (2) independent evaluators who have demonstrated ^ompetenc^ and 
experience in this type of under^taking. 




11,2 To Develop and Refine a "work Evaluation ^ 

, • i*» 

An Important outcome of the VICI demonstration will be^'the production of 
tanftiWLe ana valuable physical Improvement to local communities. The growth of 
public^mpioyment programs has added urgency to developing reliable measures" 
of the dollarvaluB of^work performedo In order to address the effect of VICI 
in monetary terms, a practical and workable methodology must be developed. To 
date most work valuation procedures have relied upon expensive and time consura- 
^ing fiscal audits or' have Utilized cost/effectiveness formulae involving ad- 
mittedly Arbitrary and subjective components. 

CPPV, has designed arid will evaluate, a fairly straightforward method for 
determining the value of V;CI work products. For each VICI work project, local 
estimators wil* compute" an estimate of the cost that an alternative supplier, 
specifically a typical private contractor, would charge for producing the same 
product. The price that private dontractor would charge will be equated with 
"dollar value** of a VICI product with possible adjustments made to reflect dif- 
ferences in the quality of outputSo 

CPPV, through its subcontractor, RBS, Will select an independent esti- 
'nator who will visit VICT sites and compute estimated alternative supplier costs 
for approximately 200 randomly selected work projectSo These iddependent^esti- 
rcates will be compared to local estimators, figures in order to determine the 
degree of agreement .or concordance. If systematic error is apparent with re- 
spect to local estinates, correctian factors will be used to adjusr figures 
across all estimates. Assuming that there is acceptable agreement between local 
estimators and independent estimators, dollar values will be assigned to VICI 
projects. T:\e dollar values will reflect what. the same work product would cost 
if. it were supplied by a private contractor. Taxonomies .will also h* constructed 
to explord* '/nether dollar value differs significantly across various types of 
work projects (e,go, rehal^ilitation vs. repair)^ 



* Work Evaluation Methodology" in thi^ conteyt has been operationally defined 
as attributing a dollar value to the outputs produced by VICI participants. 
*'Work valuation** is perhaps a more accurate' tferm In describing the nature. . 
of this ^ask. 

X 

CPPV has selected Research for Pettier Schools (RBS) to conduct the quanti- 
tative portion of the VICI research. ^' 
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II. 3' To Compare. the Post^Program Effectiveness of Youth Partlclpanting in 
thr Replicated Projects with Those for a Comparable Group of Younp 
People of Similar- Backgrounds Who Have Participated In Other Manpower 
Programs 

The key question to be addressed in thl-^^^f^ll^Ls: "Does participation 
in VICI lead to^etting a better job than participation in other manpower 
f programs?" tlues to this answer^will be obtained by simply comparing VICI~~tenSIna^ 
tion data with the termination data gleaned from other YCCIP formula programs, 
and HUD demonstrations that may be operating at each locality ♦ CPPV, has asked 
RBS to gather existing data from other YCCIP and HUD programs that resemble 
VICI •in each Site, and to present this ^ata in a descriptive, summary fashion ' 
in order to compare VICI and other programs along such dimensions as: 

. youth characteristics . 

• • nature of work performed 

• strength and extent of linkage systems 

0 

• retention rates ' \r 
\ o cost per positive termination > 

o post-program placements 

In addition, it is anticipated that this inter-program comparison effecj; 
will provide information that will aid DOL in assessing the effectiveness of 
different kinds of delivery systems in mounting prograits of this type. S{)eci- 
fjcall3> cc:?parative delivery system data will be presente|!l relative to the use 
of: 

. an internedi'ate unit (CPPV) 

• conrtiunitiy-based agencies (HUD) 

. prin«e sponsors , ^ 

It was recognized, however, in order to draw confident statistical 

inferences, about the impact of Jf^I on youth, a research design would have to 
be developed that would better assure equivalence between VICI and comparison 
youth. 

Indeed, unless there Were an equivalent group of yfrting persons with wfiom 
to compare VICI pa^ticipants, it would be impossible to confidently attribute 
"outcomes" to participation in VICI since a number of other plausible explana- 
tions Qould account for outcome patterns (e.g., local economic conditions, the 
type of yoatrfi involved in VICI.) However, the generation of an equivalent corn- 
pa risonv-gfoup has been a problem which has continuously plagued manpower research, 
nnd pe4;haps nil social services is^search. In addition to jpjogrammatic problems, 
scrioii>^ Mthical problems nritse \A\en one attempts to launch a "social experinent." 
f:f'PV..st af f , ns voU <ir. Us VICI research subcontractors (RBS/ARIJ) * and private 
lonsulrnnts, v/orked fi)r M»V(-r.il months In ordur to come up wLtfi a tlcsj^^nChi ch 
would satisfy rosearch criteria an well ns ethical nnd prograimiiatic standard;>. ' 

tvn has approv(-<l Ui;s' seUctitui ol As^;ociateb for Researcfi in BehaC'ior (ARB) 
as a sulx oui ra< r:or tf) p^^rform the follow-up intcrviowii for VICI resciiUCh, to 
^ mainf'iMi ihr M.^pu it r i x.rd inroi...Lion base, and to offer ongoinp, input rc- ^ 
/'.ar<li'nK tht* VlCf rescatth. " ' , 
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Consistent with DOL instructions, CPPV presented the design to the ap- 
propriate representatives from each VICI site as an option, not an order. Seven 
(7) of the eight (8) approved sites opted to adopt the "CPPV design" as presented 
or with minor variations. Our'design is explicitly described in Appendix B. 

JII.^ TIMEABLE OF RESEARCH ACTIVITIES 

MorcT:Tian half of tne irrglTr t^)TVrcriyr0grinns have^begun enrolling cvw^ 
didatcs. All sites should iTe operational by mid-October, 1978. CPPV has de- 
signed a comprehensive set of uniform data collection forms and manuals for 
use in all VICI programs. Data from this information base, as well as from 
the process evaluation, will be incorporated into brief monthly progress re--^ 
ports and extensive interim reports. The final research results will be avail- 
able in May, 1980. ^ « ' 

IV. KNOWN. ANTICIPATED >^^LtKELY RESEARCH LIMITATIONS ' 

In addition to those potential limitations, which ail research efforts 
encounter (e.g., accurate and complete filling out of forms, being able to 
find and interview sufficient youth in the post-program follow-up), thfe main 
limitations of this study may occur around generating an equivalent comparison 
group. The design that was earlier mentioned assumed that VICI programs could 
recruit and screen almost three-hun4red (300) youths within about a four-month 
period. Preliminary feedback indicates that problems may occur in finding 
sufficieat youths. In addition, the added screening burden may cause one 
or two sites to abandon this design, because they lack sufficient intake 
/resources. 

Should problems like these arise and scuttle the design in certain sites, 
CPPV'^may shirf t its follow-up resources to tracking youths wlio have participated 
in other Manpower programs without going through VICI sci^eening. Admittedly, 
such a fallback position jeopardizes the equivalence or comparability between 
groups since othejr programs may have screening processes or programmatic goals 
which differ from VICI. However, comparisons of this nature should yield clues 
as to the impact of VICI compared with other manpower programs. 

V. ANTICIPATED RESEARCH FINDINGS ' ^ 

Since data collection is in its embryonic stage, few directions or trends 
can be discer,ned at this time, which would give evidence for anticipated find- 
ings. 'One' critical area wherein a specific finding is apparent relates to the 
replicating process. As noted above, all VICI applicants were required to meet 
a 55et of fairly arduous program criteria, fa mosL sites, this called for the 
forging of new inter-agency cooperations. Union Involvement, and securing sup- 
plementary funding, were especially viewed as^ potential stumbling clocks. How- 
ever, of the acceptable applicant.s (13), * .eight (8) sites have met all criteria, 
and one (1) site is still in the running. This lends strong evidence to the 
feasibility of replicating even those model programs which make strong demands 
on local linkages, procedures, and other resources can be replicated. CPPV is 
paying close attention to the five (5) programs that could not meet requirements 
ill order to document obstacles and formulate strategies that might enhance re- ^ 
pi ication. ^ ' ^ 

* DOL eliminated two (2) of the original appl&ants for reasons not dfrectly 
related to VICI. 



VI. WHAT DOES THIS ALL -MEAN TQ YOUTH POLICY PRIORfTIES ? 

If the VICI research yields certain findings it is possible that these 
could be the basis for several policy .and procedural changes. For example, 
little evidence has been gathered on program replication. The terra keeps crop- 
rping-iip and inti^ently seems to make sense. However, to date, there is a pau- 
city of hard data in this area; Should,*^ VICI demonstrate that replication is 
not only viable, but economically practical. and effective in aiding youth enter 
th^ mainstream of employment, it is possible that planned replication may gain 
increased prominence as a program planning and implementatipn policy. 

The research that addresses al^rnative delivery systems (prime sponsor, 
intermediate up^ts, and community based agencies) should yield preliminary 
evidence \s-T.o the efficacy of each variation. On the policy level, such evidence 
could influence future program implementation strategies. 
i 

^. <^ 
On the procedural level, work valuation has been a problem which has be- 
fuddled economists, program planners , and , policy makers alike. CPPV's methodo- 
logy i should it prove valid, would provide a long-needed mechanism* that would 
yield extremely valuable information to policy-makers^ planners, and admini- 
strators for purposes of program development. j ^ 



VII o WHAT DOES THIS ALL MEAN<EQ\R£SEARCH? 

In terms of adding to fast-growing body of research knowledge, the VICI 
praject could pjrovide substantial offerings. First, VICI will constitute one 
^of che very few replication efforts that was well documented and evaluated. 
This, in itself, will contribute significantly to a better understanding of the 
replicating process. Second, the "CPPV design" i-s- one that has not, tp our 
knowledge, appeared in the literature. Yet, given the real world of the pro- 
gram o^rator, this design seems to strike an acceptable compromise among the 
deT.anJs of research methodology, ethical considerations, and program operations. 
The design may be especially useful in programs within the human services sector, 
a'nd could possibly constitute an advancement in action-oriented research. 

Thirdly, and perhaps most ^Importantly, -the VICI data collection forms /tnd 
the "CPPV design" were riot conceived in VACUO by a "bunch of researchers who 
don*^ know what running a program is all about." Both the data collection forms 
and the design resulted from intense interaction and numerous meetings among 
CPPV research staff and field representatives, DOL national and regional staff, 
prime sponsor staff, front-line program staff and youth. To be surejthis made 
the, design process more time consuming and at times somewhat f rustrcJxing. Hoxv- 
ever> CPPVoelieves that such a process served to {produce not only better pro- 
ducts, but:^ tended to engender genuine feelings of involvement and conrajitment 
to the research efforts of VICI. CPPV'Hill document this process and explore 
its effects. From a planning perspective, it is possible that this process can 
flttferja model to future research efforts wliich like VICI are dependent upon the 
cooperation, effort, and commitment of literally hundreds of people if the data 
is to be property defined, recorded, transmitted, stored, analyzed and promul- 
r.atnd. ' 
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' appendix a 

ESSENTIAL AND REQUIRED llODEL PROGRAM ELEME Nff^ 



1. Participants 16-19 years old wlio are Unemployed, underemployed, economically 
disadvantaged and confronted by severe difficulties in obtaining access to 
jobs. 

2. Work projects which provide both needed physical community improvements and 
varied work and training for participants. The work projects must be comple- 
table within the life of the demonstration; and the vork must be of a kind 
whichs^would hot be routinely done iri the absence of the program. 

.3. Project participants recruited from" the ^eas where youth work activity will 
take place. . ' • • ■ 

4. Well defined work crews led by skilled supervisors and a supervisor-parti- 
cipant ratio small enough*' to permit skills training. 

5. Linkages of public and private agencies with clear assignment of (and agree- 
ment t^) roles and responsibilities to include: 

(a) A loca^l. management agency with th'tr^experience and capacity to conduct 
and manage the program. ^ ^ 

(b) A youth referral agency with demonstrated access, to experience in 
working vich youth who are the .target population for this progcam, 

(c) A wbrk providing agency with the demonstrated capaQity to provide a 
suitable , inventory of projects and work orders in a timely and contin- 
uous fashion. • • - * ^ 

(d) Labor unions and trade organizations cooperating i,n the referral of 
% journeymen instructors and provision o£ apprenticeship training credit 

*for youth who work in the programt 

(e) Educational institutions which can* provide youth participants with oppor- 
tunities to receive a G.E.D." or enroll in other form^ of continuing 
education* 

6. Provision for obtaining licensing approval and/or permits to undertake phy- 
sical improvements work, inspection of completed work, and valuation of 'work 
performed. , . 

7. Provisions for post program training, education or employment. 

8^ A commitment to provide^ the necessary data and information to meet the re- 
' search rcquirementr, of the demonstration. 

' '9. Local funding to supt>lement *tho amount provided by DOL, particularly for 
building materials and supplies. < 
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APPENDIX B , 
; CPI^DESIGN 
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The proposed design calls for 60 participant slots. In addition ^he VICI 
stnicture calls for two cohorts of participants at each site. The first sixty 
will remain in VICI for up to twelve months but no more, A second cohort or 
wave of youth will be enrolled after the first wave has completed VICI. Thus, 
there will be at least 120 (60 60) participants* Assuming a 25% or so drop- 
out rate, it is predicted that there will be^ about 150 (60 + 60 + 30) VICI parti- 
cipant's per site, ^ 

Because of the urgency involved in getting VICI up and running, the "CPPV 
design" opts not to dejay intake uptil a sufficient pool ife built and applicants 
can be randomly selected for VICI (i.e., the classical experiment) or selected 
via other procedures "from such a pool (i.e., quantifiable scores/ratings.) 
Building up a large enough pool before VICI start-up would simp Ty push back the 
schedule inordinately. Therefore, thef "CPPV design" calls for programs select- 
ing the first youth cohort (n = ^ and several waiting-list youth (between' 10 
and 30) according to the program'^ own procedures and own time frame. 

After the first VICI cohort is selected along with a small waiting list, the 
"CPPV design" calls for each program to continue intensive intake for several 
months until an additional 220 or so youths have been found who meet all VICI 
criteria (both DOL's and the local program's criteria.) These youths will have 
undergone the c::act screening as the first cohort and will be ^signed, by 
lottery, a place on the VICI waiting list. It is predictable that the first i 
60 of these 220 youths will be selected for VICI before the demonstration is / 
qyer.^ It is equally predictable that those youths who are placed towards the 
end of the waiting list will not get into VICI during the demonstration period. 
Approximations of the chances of getting into VICI can be made for those youths 
who are toward the middle of the waiting list. The "CPPV design" calls for 
honestly and promptly informing the youth of his/her odds of getting into VICI 
as soon as the youth has a waiting list number.\ This decision, namely, to 
be totally honest With youths ^ is called for in order to avoid undue raising 
of the youth's expectations » For the youth who has little chance of getting 
into VICI, or for the youth who may well get into VICI but chooses not to wait, 
full effort will be made to place the youth in another^ appropriate employment/ 
training program as soon as possible. CPPV^xpects that a sizeable number of 
youths will be placed in ongoing programs. It bears ndtiuf^ that the CPPV 
de5?Ign neither directly nor implicitly encourages the formation of a "no treat- 
me nt Control group" whose^ waiting list position indicates Htftle chance ot VICI 
ci Ttry. There must be a reni cli'ancc, not just a sham chance, o^ receiving al- 
Tnniative manpower services. To better insure this option, CPPV has mandated ^ 
that no site wherein there is not sufficient potential to placa youths from the 
waiting list into alterliative CETA programs Vlll be permitted to use the "CPPV 
design." ' 

CPPV,' through its research subcontractors, RBS and ARB, would follow-up eighty 
(80) VICI participants and an equal amount of waiting lid<vouths w^o did not 
get into VICI. Follow-up will be done. 'at a standardized tinfe-. For example, 



APPENDIX B (Con'd) 



three months after a youth terminates VICI, both this youth and a youth selec- 
ted by chnnce from the waiting list will be followed up. It is foreseen thai 
this ilcsign will yield a comp»>rison -^group that is sizeable enough for conduct- 
ing statistical analysis and boasting a level of homogeneity with VICI youths 
I hat is adequate for drawing confident conclusions. Homogeneity between VICI 
And comparison groups will be double-checked by examining scores from a stand- 
ardized test of ability which will be administered to all VICI and waiting 
list youths. 
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Three agencies might appear to have some stake in this 
YCCIP demonstration project: DOL. and HUD because this demonstra- 
tion project is- being formally operated -under an. interagency ♦ 
agreement, and CSA because it has provided money for the opera- 
tion of the project. According to the DOL Knowledge Development 
Plan the evaluation of this demonstration project is to determing 
whether: (a) CSC's can more effectively link youth Community. 
Improvement efforts to other funding sources and, if so, 
whether there is a multiplier effect reflected in project out- * ^ 
comes: (b) CDC's can facilitate particular productivity by 
youngsters which is not generated by formula funded projects: 
and (c) the nature and value of project accomplishments which 
distinguish the CDC efforts from those of f orrtula-f unded projects. 
A YEDPA document dated 9/11/78 changes the second item above to 
"value of community improvement work*' and the thiyd item above 
to "the impact of. these projects on communities and participants 

in comparison with that for YCCIP formula-funded projects at 

..3 

CETA prime sponsors. 

hud's objectives differed from those of DOL. They were to 
give priority to (a) impact on thfe CDC, (b) impact on the community, 
and (c) impact on the participants. ^ CSA's objectives are not 
clear. CSA representatives never attended any of the briefing 
meetings in Washington which were attended by the evaluators and 
HUD and DOL representative. Because of the structure of the 
evaluator's contract wi}:h HUD (and not with DOL or CSA), HUD 
determined the basic priorities of the evaluation. 



2. U.S. DOL Emplbyment Training Administration, Office of Youth 
Programs, "A Knowledge Development Plan...", 1977. 

3. "YEDPA Knowledgement Development Project Funding as of 9/11/78." 

4. HUD Carant No. H 2881-RG Contract ' .C" 



The project design and methods of analysis are in two parts. ^ 

First, there are three site visits of about two weeks each to 
five of the ten demostration projects.. These are to observe 
the programs, View the work done, .interview operators of both 
HUD and formula-funded programs, interview community .leaders 
and youth participants. Thp second part of the-^'^udy is to 
^nalyze data from YCCIP intake forms, ten^inai:ion forms and a 
small sample of participant int^rvlews^^or^^ perceptions of 
the programs. We hope to compare data for both HUD and formula- 
funded YCCIPs but it is not clear whether we shall get access 
to all appropriate formula-funded YCCIPs bedojase they are under 
the auspices of DOL whereas the evaluation is under the auspices 
of HUD.* 

Among the major limitations of the study are 
the missing items on the YCCIP application and termination forms. 
Because we are funded by HUD, we are unable to off^r any incentives 
to the DOL formula-funded YCCIPs to cooperate '^or to improve data 
quality. Because HUD/YCCIP operators know we are reporting to 
HUD, they have generally been very cooperative. 

On the positive side, the amount of ^ time w'e were able 
to allocate to field research at. each site has meant that we 
can arrive at a detailed description of each of the* five sites 
we are visiting and adequately . describe the impact of these 
demonstration projects on the CDC, the community and the par- 
ticipants.' , 

Nevertheless we have come to come clear but still tentative 

5 

conclusions within the limitations %iiscussed . ' i 



1 — ■ — 

5. 'The djmonostration projects included in our site visits were; 
Sout;h Bronx, N.Y., San Antonio, Atlanta, LA., Mississippi, and 
Boston. CF., J. Ostrower and R. Feldman, "Evaluation of the HUD/YCCIP 
Demostration Projects, First Quartly Report",. Boston University 
June, 1978. 
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!• The programs are working and in most cities- useful work 
is being completed. In some cities 1:he programs are well organized, 
work is going smoothly, and participants are pleased. In most 
others, work is going relatively well but the programs are having* 
some organizational problems. 

2. There seems to 'be a trade-off between institutional 
impact on the one had and participant and community impact on the 
other. The greatest institutional impact of the HUD grants 

will be oh those organizations that have the least experiei^ce 

and consequently have less impact on the community and part3> 

cipar/tS'. Conversely, those orgstnization wi*^ high levels of 

dSjperience and a sophisticated institutioha jSirfJ^^^rfi'structure 

will be more able to have a significant community impact (e.g., 

number employed and skill developemnt) . These distinctions 

lead to the suggestion that the administration of the program 

, • * 

should be resnonsive to thb^e differences. Potential'^fbr ex- 
pansion seems ^directly related to previous organizational ex-^ 
perience. Those organizations with the leaS^onstruct ion -ex- 
perience need to do the most planning. 

3. Those programs that are faring best have had construc- 
tion experience^ hcrusing experience, and are able to comiDine 
both the youth and work aspects of the program with little 
difficulty. 

4. The programs tl^t have the most impact on the organi- 
zation are those that either change the direction of the organi- 
zation or substantially increase the organiations resources. 

5. Those programs that seem to have the most short-^erm 
impacts on their communities are those that have decided to 
concentrate "resources in the most limited geographical areas. / 

6. As expected, among the most important participant 
impacts of the HUD project are the receipt of money by the par- 
ticipants and the work experience for these youth, many of whom 
have never* been employed. In addition, many of these jobs demanded 
the kind of work perjjormance comparable to unsubsidized employ- 



ment and provided significant skill training. 

7. The job supervi^l^rs are particularly influential i.n 
working 'with youth. Union journeymen in particular are highly 
skilled and §eem to serve as economic role models. Examples* 
can be found_of supervisors who helped participants get unsub- 
sidized -jobs and they have also helped a limited number of parti- 
cipants to enter the uijions^"^^ 

8. Obviously, programs where participants are engaged in 
complex work (e.g., housing construction and rehabilitation) 
are providing greater opportunites for akill acquisition fe^an 
those with relatively simple tasks (e.g., lot clearance and 
building demolition)'. At this point impact seems significant 
where partsicipants a*re engaged in complex tasks and qraftsman/ 
participant ratio is not high. Where this ratifl has been 
observed to be\high, e.g.,. 1:10, a variety of problems seem 
evident. These include: a) slower skill acquisition, b) njore 
discipline problems on the job, and c) a, lower likelihood of the 
supervisor providing an effective role modql. 

j '9. Although 'it is still too early' for definitive comparisons 
with ^he formula funded YCCIPs', the following observations can 
be made: a) In general, the HUD/YCCIPs are substantially larger 
^than the comparison groups. The larger scale of these projects 
and more limited definition of "comiaj^nity " predisposes questions 
.abouth community impact in favor of the HUD/YCCIPs. b) Where we 
have been able to make site visits at the comparison projeo.ts, 
it appears that the HUD/YCCIPs are having a larger community 
impact. Obviously, these conclusions are still based upon interim 
observations and interviews in. the cpmmunity. They will be pursued 
at greater length during future site visits. 

10. Given the current state of our research, we believe it 
is important to develop ideas on how neighborhood units can be 
integrated into an overall employment strategy. 

One important^ purpose of an interagency demonstration is 
probably to permit the substantive priorities of the second 
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agency to impinge on the demonstration project. It is probably 

reasonable, therefore that the evaluators take * serious account 

of these priorities and depart somewhat from the ojectives of 

the primary agency (DOL) .*^0n the other hand, commtinication 

between agencies with regard* to DOLs Knowledge Development Plan 

could have been better. 

* \ — 

Estimates of pro jec^operators of the likelihood of project 

renewal probably has an important effect on their administration 
of the projects. Their belief that the demonstration will end 
in six, four, or two jnonths affects the programs during their 
critical final phases. Basic activities of the programs such 
as finding new work, participant and staff morale, job develop- 
ment and departure of staff are affected by preceptions of when 
the project will terminate. This is a critical input. for A 
demonstration project designed to last only one year. ,These 
depons6i:^tions are probably of too short a duration to answer 
many of the questions of both HUD and DOL. In the absence of 
factural information, demonstration project directors >ave already 
made judgements of when the 'project is likely to terminate and 
are acting accordingl/y . This factor needs serious consideration 
in the design gf program dura^on and in the design of evaluations 
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For many low-income teenagers, the gap between school and work 
seems to be widening. The rjumbep of years between leaving school and 
finding a full-time, ''adult" job is increasing . * The kinds of pre- 
paration provided by the school for the "world ,of work" seem to be 
getting even less relevant. Very little is really known ^bout how 
schools can effectively' combine school ,and worTc experiences to 
facilitate future employability . Even less is known about which 
particular -in-school models and treatments work. well, and why. 

The iDepartment of Labor,, as a means of encouraging 'innovatioli 
in school-related youth employment programs, has launched a special 
exemp 1 a ryJji- school demonstration project. This demonstration project 
„is identfrying some 55-60 program models in four primary areas of 
focus; y (1) Career Information,, Guidance, and Job Seeking Skills, (2) 
Academic Credit for Work^ Experience , , (3) Expanded Private Sector 
Involvement, and (4) Job Creation throxigh Youth-Operated Projects. 
The goal in narrowing the demonstration to primary areas of focus . . 

is: (1)/ to identify key areas in which more knowledge is needed-, 
then (2) ^o support enough program? models in each area, and (3) to 
learn a lot^about each area from an analysis of a' variety of related 
program models. , 

. Tb help the Department of Labor select appropriate program 
models, provide tecfinical assistance to funded projects, coordinate- 
what can be learned from the demonstration, and prepare "how-to-da-it" 
guides for in-school employment related efforts, a non-profit, 
.intermediary corporation was formed in the winter of 1978^. This 
corporation, called Youthwork, Inc., has undertaken the responsibility 
for insuring that' we learn as much as possible from what the demonstration 
projects are trying. . ^ — yr-^ 

. , Youthwork *s special mandate froip the Labor Departmeptl:!^ to focus 
attention on the employment ^problems of ixk^school ^yoyth,' on the 
capacities of educational institutions to%ddress these problems, 

-and on* critical issues emerging from the evo-lvinq relationship^ 
between, the educational and CETA systems. With regard to 'the problems 
of in-school youth. Congress Wants to know about (1) the feasibility 
of, and procedures for, implementing, expanding, and a^^pting 
exemplary programs for in-school youth ahd returning dropouts, 
the number and characteristics of youth served by these programs 
and- of those rejected, (3) tl>e kinds of experiences, training, and 
other services being provided to youth, and the kinds of work being 
performed by them, (4) the ^time spent in programs, aiid--(5) the 
costs of variouls programs and approaches. Youthwork 's knowledge 
develc^pment plan places high pribrity on obtaining this information.. 
Gradually, Youthwork has also developed a knowledge development strategy 
designed to maximize what will ultimately be learned. This paper is 
an eff^^rt^to describe the outl^ines of that emerging strategy. 

^ The strategy begins at the^ver^y beginning. It assumes that so 
little is currently known about effective work-education combinations 
that few working hypotheses are justified, intense documentation As 
required, and ^n initial knowledge and data base must be developed. 




It attempts to address -the following^broad questions : v 

1. Can th^ school become a more valid institutional instrument for 
preparing low-income youth to mak^e successful transitions to *the 
world of work?^ 

2. ^ What changes (necessary institutional modifications) have' to 
occur to help the 'school become a more valid institutional instrument 
for p^^eparing low-income youth to make successful transitions to the 
world of work? * 

3. What are the most effective techniques and strategies to develop 
closer cooperation and coordination between the CETA system and local 
educational agencies? 

4» What are the most useful available mechanisms or strategies to ^ 
attract and maintain more intensive private sector participation in 
in-school youth employment programs? 

5. Do the '"exemplary" progr^ma included in the demonstration have the 
power to impact* differentially upon the educational direction, 
trainiYig opportunities, and early employment outcomes of participants? 

6. How (or why), do the "exemplary" programs impact differentially 
(i.e. do 'a better job) upon the educational directiori, training 
opportunities, and early employment outcomes o£' low-income youth? 

7. What is the relative impact of a variety of different treatments 
and models with similar goals, in each, of the ^our focus areas? 

8. \ What. are the particular components of- programs which. can be * 
isolated as most critical to-increased opportunities and other 
^successful outcomes? ^ 

9. What effective techniques can be developed to link what is learned^ 
through job-site experiences with academic credit leading to a^ 
reqognized degree? ^ ' ' 

10. How can youn^ people be more directly involved in the creation 
of their own jobs, and in the planning and operation of employment 
programs ' de'signed for their benefit? 

Ideally, we would want to know the ultimate impact of the various 
programs on participants^ employability , future education and employment 
records, income levels, etc., but Youthwork's knowledge development 
plan recognizes, as does the Labor Department's plan, that the ^ 
practical limitations of time, money ^ and research capacity in a oner 
year demonstration program are very rte^l; and that such goals are 
probably impossible to realize. \^ 

Instead, the strategy focuses heavily on the developmental process 
within each project: how it evolves, where it accomplishes anticipated 
outcomes, how it accomplishes anticipated outcomes, what are the 
characteristics of each participant at the beginning; what changes * 
participants experience, etc.. 

' ^ If 
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This information w^ill be obtained' through a three-tiered, hierarchical 
knowledge development system: (1) local management information 
comipohent, focusing on the individual data which will be r^ularly 
fed into Youthwork's centralized data bank; (2) the execution of 
.a local knowledge development plan, sponsored on-site by each funded 
project, with techniqal assistance, advice, and coordination 
from Youthwork; and (3) a -national, cVos-s-cutting analysis of 
funded projec^, designed to shed light on several mijor issues: whether 
(and how) tl^j^School can be an effective instrument in preparing 
low-income y^th to make success'ful transitions to the world of 
work; the means for incjceasiTig private sector participation in ih-school 
youth employment progr^s; whether (and how) demonstration programs 
improve low-income youth's prospects** for better employment outcomes, 
future training oppo«tunities , or future educational |)pportunities; ^ 
and whether (and how) steady progress can be made in promoting ^ ' < 
closer cooperation between prime sponsors and local education agencies. 
The three-tiered approach is designed to have a natural logic of 
its own, in which the uriit of analysis gets broader as one moves 
from tier to flef~rindividual participants, local projects, national ^ ^ 
demonstration) . The ultimate objective is to facilitate making 
generalizations across projects about 1 the valu-e of various treatments, 
or approaches. 

Each of the thr.ee levels/ of inquiry has a corresponding method 
of data collection, analysis, and reporting. In addition, the three 
levels combine to cross-validate certain impressions apd to form 
a common ppol of data and conclusions that will be used to answer higher - 
order policy, concerns. Inquiry at the policy level will be conducted 
with a case study methodology. Data for a case study of key projects 
will be collected by field rese'archers , using structured o bserv ation 
and interview/ techniques. In addition to fornfing th'e basis for~~''" - 
individual case ' studies , .these data will be used to cut across 
the various projects and focus>areas to make judgment about pressing 
policy issues. / 

The intermediate level of inquiry is built around local knowledge 
development ^plans . Where programs have used, copunon designs and treat- 
ments, Youthwork has asked that common measur^ and standard instrumenta- 
tion be utilized in local knowledge development plans. These common 
measures will be. used to verify project results and to conduct 
special across project examinations to determine if there are differential 
impacts on the various populations served. Another major component 
of this level of inquiry will be a third party summative evaluation 
of each project. These evaluations will be synthesized to make^ " 
judgments about various program treatments and designs. Data from 
these reports will be synthesizfed • to prepare* a series of policy 
reports. 

The third level yof inquiry. will gather data from a series of 
one-time-only, reports to be submitted by each project. These^r#orts 
include reports on start-up, unique features,^ outcomes and preset 
costs. Cross-cutting analyses of each of these reports J^7ill be 
conducted to determine the most effective program designs and 
treatments. Conclusions from these analyses will be synthesized in 
a series of How-To-Do-It Guides and poJ.icy reports. 
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ilow~To-Do-It guides will focus on such specific issues as 
how local programs can conduct effective self-studies; how to involve, 
young people effectively as peer counselors and job pOTcTcement 
specialists; how to tailor on-the-job learning to facilitate the 
maximum development of writing, reading, and mathematics skills; 
how to reduce barriers to ybuth employmqjit in the private sector 
through intermediary organizations ^hich deal with bonding, 
insurance, paperwork-, and other "overhead problems" of the employer; 
or how to organize the' financial aspects of ^ youth-operated projects 
so that earned income can be utilized for project expenses. Policy 
^reports might .cover such questions as what responsibilities the 
school systems should have for the education and training of young 
people and what responsibilities are more appropriately handled 
through another system, such as CETA. 

National knowledge development activities, began before the. funded 
projects were known, with the initiation of four kinds of activities: 

(1) Knowledge development "agenda - setting conferences . in pach 
of the four key focus areas, to help guide early decisioni 
regarding areas of emphasis for local knowledge developrnaftt ; 

(2) . Reviews of research and literature in each of the .focus 
areas; 



r3) The creation of, how-to-do-it guides regarding local knowledge 
development in each foci;is area, so that each funded project 
" could "have, at an early point in their history, clearly 
\ written technical assistance manuals regarding how to learn 
as much as possible about what they were trying; and 

(4) A corporate self-study, conducted- by* an outside third-part; 
observer, regarding the decisions made in the development of 
Youthwork's activities, the factors influencing the, etc., 
to provide on-going data about the progres^s of an intermediary 
organization created for the purpose of implementing a Labor 
Department demonstration. ^ 

Other knowledge development activities currently planned include: 

(1) How-^o-do-it manuals : As funded projects move into their 
implementation phase in October of 1978, information will be 
systematically .gathe]3ed regarding components ^^©aa^iC'ritical 

to successes. *Th,is information will ultimately be synthesized 
in how-to-do-it guides in' each of the four key focus areas. 

(2) Policy summaries : Each of the fmT*§d projects* will produce 
one-tim«i reports on\start-up problems, implementation issues, 
outcomes, and-costs. These reports will be syntji^sized in 

% four policy summaries, which wiil be* published periodically. 

The first report (on start-up problems) will'be ready in. 
"^'ebruary or March, 1979. * ' ' ' , ■ 

(3) Special studies : Special studies will be commissioned on an 
as needed basis, |pcusing on studies which can be put to 
immediate use by local program, managers and polic^planners . 

^ 'These studies should be completed by July or August, 1979. 

♦ « 
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Although the Youthwork knowledge development plan is ambitious, 
the limitations and impediments to learning *f rom the demonstration 
have also been recognized. This is not a controlled experiment, nor 
even ^n uncontrolled experiment. It might best be described. as an 
Uncontrolled non-experiment. It is not controlled because it was 
conducted as a competition involving many relatively unsophisticated 
program operators. It is not an experiment because the program 
operators have .each formulated their program design with only occasional 
reference to the application guidelines; and many 'program operators 
Jiave demonstrated little or no interest in experimental desi-gns. 
Many paid only lip service to knowledge developipent in their original 
proposals; dnd negotiations i^garding k^nowledge development have 
had to be conducted hurriedly, in ordep to keep ^to Labor Department 
funding schedules. 

There is even some considerable question t^egarding the "exemplariness 
^f many funded projects. Although the program is referred to a^ a 
demonstration of "exemplary" in-school projects, it cannot be taken 
for granted that most programs will actually prove to be exemplary \^ 
in their implementation. This is an issue which must still be studied, 
as all the others • This creates problems, for example, in thinking 
about "planned variations" across sites. It is a lot more difficult 
to accomplish planned variation among non-dxemplary programs. 

Another limitation springs fr*^m rq^strictions on amounts available 
for local knowledge development. Although the program is labeled a 
demonstration, regular CETA restrictions on administrative costs ' 
(20% maximum) have been applied; and knowledge development is defined 
primarily as an adminstrative task, since more thari^lO% has usually 
been allocated for , administration by prime sponsors and pro.gram 
operators, less than 10% is usually left to pay for local knowledge 
development costs. This is another example' of what the Labor ^ ^ 
Department referred to, in its original (1977) knowledge development 
plah, as the tension between getting the money into the pipeline 
and carefully orgafTlzing to learn as much as j)Ossible from the funded 
programs . 

All these limitations argue for the strategy adopted at the outset. 
To the extent possible, resources have, been concentrated and structured 
so that underlying ideas can be given a reasonable test. Youth^^brk ' s 
national knowledge development efforts have been focused on finding 
out how to dOo a few key ^things better. We have alsiempted to direct 
(or redirect) a, major emphasis of each project's knowledge development . 
plan on an issue or issues identified as national priorities. If 
all goes well (a big "if"0 , valuable information should be available, 
for future reseaiy:hers , program operators, and policy makers, within * 
twelve months.. might discovW, for example, that jobs can be 
created less expensively through youth-operated prqjects than through 
public sector employment. Or that jobs created in youth-operated 
projects provide • a more valuable 'transitional experience' than public 
sector jobs. We might learn more about how-to build basic skill' ,t* 
instruction . (writing, mathematics, etc.) into thq, on-site job experience, 
and how to measure the academic credit* value of skills learned on 
the job more accurately. We might identify techniques for (Sushion^ng 
first job experiences in the private sector, with implications for 
the expansion of private sector* Oacements. , ^ \ ^ . 



Our national study might also tell us more about the value of 
placing a certain proportion of youth employment funds within the 
' schools. What, for example, would be the policy implication^ of a 
finding that, even in "exemplary" in-school programs, institutional 
impediments make significant participant outcomes highly unlikely? 
What would be the implication of a finding that peer or para pro- 
fessional counseling makes roughly the same Impact as professional 
counseling? What would be the value to prog.ram operators of a private 
sector treatment which successfully places' small businesses in a mento^inq 
relationship with individual participants and absorbs some of the 
usual costs (in time, money and energy) of employing temporary 
help? ^ ' 

From the above, it should be clear that Youthwork is dealing 

with issues which are potentially verf important to program 9peration 

and policy-making in the area of in-school youth employment programs. 

If its current knowledge development efforts are successful, it 

will not only have provided useful ways to identify and spread the 

best of what happens in the in-school demonstration, but it will 

•have created a sophisticated data base in an area where the currently 

available data are skimpy and incomplete. This may make possible 

future research' tasks that are hard to get at in a time-limited ^ 

demonstration -^areas involving institutional changes within schools, 

changes in the definition of the kinds of people most appropiate 

for various^ staff roles, information about how to facilitate the 

. maturation process through increasing the operational responsibilities 

of young people within a project. 

4* 

The implementation problems of the first year may also argue **for a 
significant nar^rowing of the knowledge development focus in a demonstration 
pripr to the beginning of any competitive or non-competitive process, 
in recognition of the., powerful anti-research forces unleashed by the/ 
simple desire to "put a jingle in the jeans" of our nation's diadvantaged 
yputh. Or, finally, we may^at last begin to focus on the* question 
of' whether efforts which begin with the motive of putting a "jingle 
^ in the jeans" of our 'nation's unemployed youth too often end up 
^adding' to the rustle of dollars 'in the t^^o^sers of the professionals 
who staf f ^he CETA system's and local education agencies. 
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BACKGROUND: TO E PURPOSE OF THE PROPOSED 
DOL/ Nffi FIELD ffiST OP TtE CMSk MERra5GRA^l 



• The Problem and the Program ' ' ' ' 

(1) The Bji;oblem ; In 1973, over 10 percent of youth aged 14 to 24 
left school 4i0«ore receiving a high school diploma. The percentage of 
those that left school was higher for blacks than for v*iites: amcfag X 
18 and 19 year-olds, 27.7 percent of the black males and 23 percent of 
the black females had dropped out in 1973, in contrast to 14.1 per^nt 
for white males and 15.2 percent for white females. Dropping out meant 
bad economic news. According to the U. S. Department of Labor, 20 
percent of high school drop-outs between the ages of 16 and 24 were 
unemployed in 1974, compared with 10 percent of all high school graduates. 
When drop-out unemployment rates are compared by race, youth from black 
and minority groups are doubly disadvantaged: about 16 percent of white 
yout^h without high school diplomas aged 16 to' 24 were unemployed in 1974; 
32 percent of drop-out youth from black and other minority groups wanted 
nobs and could not find them.^ 

(2) The Program : A program developed by the Opportunities Indus- 
trial izationTorporat ions of America, Inc. (OICs/A) in Philadelphia, Pa., 
gave promise of showing how to turn around these statistics. The Career 
Intern Program (CIP) is an alternative ^high school for .16 to 21 year olds 
who have dropped out of school or are at risk of dropping out before 
graduation. The goals of the program are to enable these students to 
con^)lete high school, acquire occupational knowledge, plan fdr a career 
and improve their reading and mathematics skills — in short to facilitate 
the transition from high sphool to work or further education. 

The CIP itself has three phases. Phase I is directed at^iirproving 
student's career awareness. The phase- lasts 21 weeks and includes 
Classes in English (with an en5)hasis on skills mastery). Mathematics, 
Social Studies and Science. During this phase, learning is keyed to * , 
occupations through specially developed materials. Weekly' individual 
counseling sessions, combined with the exploration classes, lead to - 
development of an individual learning plan for each intern. v 

Phase II may last from four months to a full year, depending on how 
many credits interns need for graduation and rate of ^ student progress. 
In Phase II, interns participate in two to four 'Tiaflcls on" job experi- 
ences and in other courses which fuse academic and career information 
in a sequence of individual learning activity packets. In acldition. 
Phase II stresses indi<vidual instruction, independent study and advanced 
courses in the academic subjects. In Phase III, students concentrate ^ 
on the transition 'from high school to work or to more advanced education. 
Students choosing the option of vocational or on-the-job training are ^ 
assisted by counselors for six months after leaving CIP; college-bound 
interns are 'counseled for a full year after leaving. 
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• The Evaluation Design, Methods, Measures and Findings for the 
Original Philadelphia. Study 

(1) Design : The design involved three separate cdhjj^ts of appli- 
cants, applying at six to eight month intervals. Through over-nrecruitment 
and a. lottery process, known in advance to all applicants, three separate 
sets of experiinental and QOitrol groups v^ere selected in a ratio of about 
3 interns to 1 control. This ratio was selected to permit maxmim entry 
into CIP with the miniirum N estimated to be needed for a group large 
en6ugh to be sensitive to educationally meaningful effects, 

(2) The Methods : Stjtidy methods combined ethnographic studies and 
psychometric data/ The ethnographic study included (a) case studies in 
depth, over time of interns and cdhtrol students (b) semi-structured 
observations of classes and program events, and (c) field observations 
through a participant-observer • The primary purposes of these studies 
were (a) to help understand hew the student's experiences in CIP and in 

- the comparison schools differed, (b) to iiiprove the description of the 
CIP processes and practices, (c) to identify what about CIP might be 
accounting for program effects, and (d) to collect data about effects 
useful in their own right to describe program results. 

(3) Measures : Four tests were administered: The Stanford 
Achieveinent Test (for reading and math performance) , the Career Maturity 
Inventory (for career knowledge and development), the Coopersmith Self- 
Esteem Inventory, and the Intemality/Extertiality scai^ These measures 
are described in the Final Report, Vol, II, pp* 11-18; pp. 81-149. 

All tests were administered prior to selection of all applicants and 
after the program to interns and to controls. (See Final Report, 
Vol. I, pp. 9-10) 

(4) The Findings : All data were audited. Several statistical 
approaches were used (variants of ANOVA and ANCOVA) to help rule out the 
possibility Ibhe conclusions would be due to a statistical artifact. 
Among the findings: 

• In December 1975, 67% of the CIP students but oily 13% of the 
controls had graduated or were still attending school. About 44% of the 
interns had received •high school diplomas and 23% were still at their 
studies, while only 7% of the caitrol students had graduated and only 

6% were in school » 

Changes in reading and mathematics achievement showed interns 
gained more than contrjols, although the final levels of both groups left 
room for improvement. ' ^ 

• CIP interns showed very fci^ levels of career planning and deve-^ 
lopment skills, as measured by tnfe^tests, interviews, and the ethnographic 
observations. W 

• Graduates were ^followed ^p in Fall 1975. Of 77 graduates, 71^ , 
were employed, in college fo^HjT technical school as compared to 39% of the 
controls. None of the control men had enrolled in college or technical ^ 
school as contrasted to 2y/o of the male interns who were ccxitinuing their 

education. 
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These findings are described in the Final Report, Vol, I pp. 101-181 
and the data are presented in detail in the Final Report, Vol. II pp. 
19-73. 

• The Proposed Department of Labor /National Institute of Education 
Field Test ot the Career Intern Program 

The Youth Employment and Demonstration Projects Act of 1977 
(PL 95^-93) is addressed to the plight of the three million young people 
between the ages of 16 and 21 who 14:1 1976 alone were out-of-school and' 
out-of'-work. Many of these unemployed young people came frcxn very poor 
families, families who were unable themselves to cushion the blow of 
youth employnent. Many came from minority groups. Many had dropped out 
of high School The Youth Employment Act and Demonstration Project Act 
(YEDPA) has t\^70 main thrusts. The first • is direct provision of work; 
the second is learning what could be. done to have a larger, and longer- 
tetm impact on the problem of youth' unemployment. The YEDPA provides 
authority to conduct demonstration programs. 

Under the demonstration program provisions (Section 341) , the 
Departnoit of Labor, Employment and Training Administration, is support- 
ing tests of a variety of approaches to helping young people stay in 
school, conpletjB their training, and then to make the transition from 
school to work. Of particular interest is finding out whether programs 
proven effective on a small-scale continue to be of benefit to low^incotpe 
youth when the approaches are implemented on a larger-scale. Such studies 
will help identify specific programs that can wrk on a nationwide basis; 
but also the studies might identify what program elements are necessary, - 
sufficient, and reproducible and thus inform the development more generally 
of workable, effectil^ projects. 

The Career Intern Program ^me to the attention of the Department of 
Labor as a program of proven effectiveness on a small scale (Philadelphia' 
prototype site) in helping low^-income youth, who had dropped out of school 
or who were at risk of dropping oiit, complete school and make the transi- 
tion Jrom school to successful post-high school experiences. The 
Departntent of Labor has entered into an Interagency Agreement with the 
National Institute of Education to manage a field-test of the Career 
Intern Pttgram. ' " 

The Interagency Agreerrent established two conponents^ for the field 
test. The first component is the operation ot the Philadelphia prototype 
site and at least four addtional sites (three urban and one rural) by the 
Opportunities Indu^riaiization Corporations of America (OICs/A), The 
' four sites are Seattle, Detroit, New York and Poughkeepsie. A separate 
award from the Institute to the OICs/A has 'been established for the 
managemepit of the program operations. The second component is an eva- 
luation of -the field test. Both components are under the direction of 
the Nati*6nal Institute of Education « 

/ 
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• Goals and Objectives of the Evaluation Study 



A. Goals t The goals of the evaluation study are two-fold. 
The first is to learn what education, broadly defined, can do to help 
Ic^income ^youth at high risk of being uneniployed prepare for a success-, 
ful transition from high school to' work or further schooling. The 
second goal is to find how educational innovations which give promise , 
of working on a small scale can be transferred elsewhere to benefit the 
larger populations whose needs the smaller programs were intended to 
seirve. These are ambitious goals indeed for research. The field study 
may contribute modestly to reaching them; it is not expected to attain ' * 
them. 

4 

B. Objectives : There are four specific objective tasks which the ^ 
exaluati(Xi ot the Uie field test is expected to reach. .These are phrased 
belcw as quest ichs? 

1. What happens to the Program itself in the process iirf^lemerl- ' 
tation in additicyiai sites? What accounts tor the changes or adaptations , 
if anyY For the fidelity^ if any^ to the original program goals and 
practices (process evaluatioi)? !toitormation ai* the implementation of CIP 
in the new sites is intended to help answer three sub-questions: 

• does the program remain the same in terms of goals and 
practices? 

• are the changes, if any, improvements in terms 'c3f the over- 
arching goal of effectively helping young people who are 
drop-outs, unemployed or at risk of dropping out? 

• is the process of implementation more effective through a 
system such as OIC in coti5)arison' to the processes of 
research utilizatiop and dissemination through the usual*" M 
developers-public school linkages? 

The answer to these questions should help assess how well and readily the 
CIP approach can be inq^lemented, a question of som^ importance with regard 
to the possibilities of larger dissemination efforts. The answers also 
should contribute to knowledge about implementation in systems which differ 
in power relationships, political consideratiais, incentives for change, ^ 
and other variables believed iirportant in the literature on educational 
change. • ' 

2. Does the Career Intern Program continue to be effective in 
helping you%^^n_J.t: is implemmtea in sites other than the Philadelphia ' 
prototype ? The CIP "worked" in Philadelphia, according to the earlier 
evaluation studies. Interns carpleted high school, improved their reading 
and arithmetic skills, iii?>roved their career planning knowledge and skills, 
and successfully made the transition from CIP to paid enployment, continuing 
education at the postsecondary level, or, voluntarily, the occupatiai of 
homemaker. Does CIP continue to work in helping -other young people achieve 
these outcomes, when the approach mwes from a small, somewhat protected 
prototype, to at least four other sites? 
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3, What happens to yotu:xg people in the CIP process that ccxild 
account for itFeTfectiveness? ITie early evaluation identified ^hat seemed 
to be essential elements which accounted for CIP's effectiveness for the 
interns, -(See Final Report: Vol I, pp. 201-206). These program components, 
identified mostly through the case studies and ethnographic research, are 
hunches about essential elements. The four new sites offer additional 
opportunities to find out if these ^^lements are in fact responsible for 
CIP's outcomes and if they are essential for program effectiveness. The 
task is to identify the essential eifements. The answer to this question 
might help in designing other programs. 

4, How does the CIP approach compare in effectiveness, feasibi- 
lity > impact ai3^ther tactors iirportait for policy with other approaches 
(e.g. Job Corps, Action sponsored youth programs^ to helping the jgopulation 
to be ^served through the YEUPAV ^fhe answers to this question, which may have 
to be quite tentative, should help cumulate knowledge about, the relative 
merits (and demerits) of different approaches to helping low-income, high- 
risk youth. 

• Sunmary of Data-Collection Methods of theEvaluation Study 

In general, there are four data-collection methods proposed to 
gather the information necessary to complete Tasks 1 and 2: standardized 
tests, interviews, observations, and ethnographic studies. In addition, ^ 
there is one questionnaire to be administered to c<xitrols. at the posttesting 
session. 

Tasks 1 and 2 : 

Standardized tests .) To assess program elements on interns, a 

battery of standardized tests will be administered iimiediately before, at 
about midway through, and after treatment. The content of the tests , 
'includes acadonic achieveirent , self-perception, logical reasoning, and 
career awareness and planning skills. 

To determine if program. climate is the same in the new sites as 
in the prototype site, standardized site climate tests will be administered 
to interns and staff members. , 

Interviews. Study staff members will interview CIP interns and 
staff numbers as welL as relevant ccmiunity members to collect information 
for both Tasks 1 and 2. ' For Task 1, data from the unstructured interviews 
will be used to create descripticns of the CIP implementations at the four ^ 
new sites. These descriptions will be compared to a model of the prototype 
program d^eloped on the bases of analysis of the program documents used by 
OIC/A staff to train nfew site staffs and of descriptive information gained 
directly from interviews with the OIC/A staff. For Task 2, interviews will, 
yield data about outcomes of program participation not measureable with ^ 
standardized tests, such as how satisfactorily CIP graduates perform m job3^ 
compared to controls^ in the perceptions of employers., ' ^ 

Interview topic checklists were designed^ to structure interviews 
as little as possible. -The study staff's experience has been that the most 
comprehensive and most accurate data can be obtained when respondents are 
not limited to a prespecified set of responses to questions. Thus the 
interview guides consist primarily of mnemonic lists to remind interviewers 
of the topics about \^ich data are needed from respondents. 

. « 
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Observations , In addition to interviews of CIP participants to 
determine the fidelity of nevr-site iraplementatiois to the prototype, the 
study will include structured observations of CIP classroom sessions to see 
if the instructional techniques prescribed by the developers are follfcwed. 
bata fron the classroom observations will enhance the analysis of CIP 
operations for Task 1. 

« 

• * Observational data collection requires no respondents, as the 
pbservers do not ask questions of those observed. They singly record 
events. 

Ethnographic studies ^ Another method of data collection for which 
no instruments are needed will be ethno^aphic observation by study staff 
members. Through the ethnographic studies, more data will be collected for 
making the judgpents necessary to x:an)lete Tasks 1 and 2. 

»> 

Ethnographic, ^participant-observer data will be recorded by 
evaluation-team staff members v^o will spend several weeks in each CIP site. 
The participant-observers will attend classes, assist teachers and other . 
staff members^. and get to knew CIP participants in a more casual way than 
short exposure allows. The ethnograpttl^rs will record their observations of 
CIP processes for inclusicn in the study's data base. 

e 

Ques t ionnaire o One questionnaire is included in the survey 
design. To be administered to controls at post test time, the quest ionr^ire 
is designed to find out what the controls were doing while the interns were 
in the program and v*iat they are currently doing, so their further education 
contacts or en5)loyers can be interviewed. 

T ask 3 entails analyzing the CIP processes an^ its outcomes on interns 
jointly to' draw inferences about v^iich program features lead to which out- 
comes. No data beyond the process and outcome docuirentation involved for 
Tasks 1 and 2 are necessary. In essence, Task 3 is an analytical process, 
requiring no additional data. 

« 

Like Task 3, Task 4 is primarily analytical <, Data will be reviewed 
about other programs intended to achieve similar outccpmes for similar 
groups of youth through secondary analyses of evaluative data collected 
byi these other programs (e.g.. Job Corps, Action-sponsored programs). 
These data will be Obtained through literature reviews and conferences 
with other evaluators. Only cost information, which can be obtained 
through document review, is needed from the CIP sites in addition to the 
data collected for Tasks -1 and 2. 
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CPPV'S PRIVATE SECTOR INITIATIVES DEMONSTRATION , ^ 

A Paper Prepared for the Office of Youth Program's Knowledge Development 
Project Conference (Reston, Va., October 5-6, 1978) 
♦ I 

r I * 

1 , Overall Purpose of the Demonstration and Research 

^ The Corporation for Public/Private Ventures (CPPV) is under contract 
to the Department of Labor's Office of Youth Programs to design, operate 
and research a multi-faceted Private Sector Initiatives Demonstration, 
aimed at. testitig a variety of approaches for increasing private s'ector 
employment opportuniti^^s for youth. 

The initiatives in this demonstration represent five different approaches 
, to the problem, as follows? 

. J' ^ 
^ 1) New Career Pathways — building access i^outes for disadvan- 
taged youth into private sector jobs through restructuring 
of entry-level positions; developing institutional linkages 
between private sector firms and public or non-profit organ- 
izations; creating subsidized training relationships between 
employed adults and youth with ^ low adult/youth ratio; and 
carefully modeling training^and^'career entry sequences tailored 
to the needs of specific firms or industries. 



J 2) Youth Pre-Employment Services — testing the effectiveness 

of existing non-profit programs which combine job develop- 
ment, placement and follow-up activities with modest job-' 
related education and counseling. 

3) Use of local business intermediary units to reduce transac- 
tional costs to firms - such as private sector-run placement 
and job development services, or OJT contracting services - 
which eliminate paper work, *'hassle" and the necessity of 

. private employers to deal directly with, government agencies 
and programs. 

4) Youth Entrepreneurship Programs — testing the viability of 
several approaches to creation of small business ventures 
which combine modest front-end subsidy (to keep them compe- 
titive) with a youth training and employment agenda, 

f w 

5) Direct ' Incentives — a formal and limited expeViment to 
test subsidy formats, ^more generous than^ existing OJT, to 
increase hiring and training of out-of-school disadvantaged 
youth by private sector firms. 

In addition, CPPV may conduct research on a program which test^ tlio 
use of volunteers (employed adults) to assist ys^th in job placement and 
adjustment. 



17C 
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The overall purposes of research are to study the impact of each 
initiative, and the programs funded within each, on youth career enhance- 
ment; .to assess and document the program development a-nd implementation 
process followed in each initiative and its component programs, both to 
provide qualitative assessment and to provide information useful for repli- 
cations of successful approaches; and to compare effectiveness (for whom and 
under what conditions are programs effective?). 

.Th>- initiatives in this demonstration are closely derived from the 
OYP Knowledge Development Plan, whic^ ' said, ' * 

A demonstration project will be undertaken to study and test 
four separate approaches to increasing employment for youth 
in the private sector: 

' . ** . 

(1) ^Direct employment subsidies will be provided for each 

disadvantaged youth hired. ^ 

(2) Training cost subsidies similar to those under the 
J0BS-C9ntract program will be tried, with varying 
levels of subsidy and specification. 

(3) Apprenticeship subsidies for smaller employers to hire 
and train youths similar to those tested in the 
Community Services Administration's exemplary Open 
Roads project will be further tested. 

^ , (4) Entrepreneurship options for youths which have been ^ 

developed in some localities will be carefully ^ 
demonstrated and tdsted . 

» / 

The "Mre-employment services initiative" was added to the typology of 
programs described in the KDP to enable close research scrutiny of a program 
model th'^ Department of Labor has already provided considerable support, 
the 70001 prograTp, and to permit. expansion under close research scrutiny 
of the well-regarded Jobs for Youth program from its New York and Boston 
bases to a third site, Chicago. The themes of new career pathways and 
volunteer involvement in job assistance emerged from discussion^ with Office 
of Youth Programs staff subsequent to publication of the KDP. 



2 . Specific Issues to be Addressed in Impact and Process Analysis 

Certain kinds of generic questions will be probed in all programs. These 
include : ^ 

A) Impact on Participating Youth 

Here the emphasis, wi 1 1 rbe on economic impacts, including skill 
development, increases in wage levels and increases in employment 
stiibility which can be attributed to program participation. 
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B) Impact on Firms • ' ^ 

Here the emphasis will be on indicators of increased employer 
} ' ^ receptivity to the hiring of y^uth workers, as indicated by increased 
hiring of youth (as a percentage of all workers), and b^ such insti- 
tutional impacts <nR alterations in .personnel policies, hiring prac- 
tices or job structures which have the effect of opening up oppor- 
* tunities for youtbi. 

C) Indicators of Program Effectiveness 

Such indicators include cost analysis (e.g., cost per positive 
termination, cost per placement in unsubsidized private sector 
employment per month); youth attendance and retention in program; 
and ability to reach and maintain projected service levels, 

D) Process Analysis and Documentation / 

( 

Under the rubric of process analysis and dpcumentation, research 
will provide thorough statistical, budgetary and programmatic descrip- 
tions of each program; qualitative assessments of program operations 
(e.g., leadership, managements skills, skills of trainers, organi- 
zational climate) and an analysis of factors enhancing or constraining 
program success, both within the program per se and in its political/ 
organizational/economic (labor market) context, 1 

Process analysis, in CPPV's view, is important not only to help draw 
policy inferences from quantitative findings but also, and especia^lly, to 
provi\(^ guidance for replication of fef f ect ive\^rogram models, ^ ^ 

' While these generic questions frame the overall research effort, 
initiative raises intrinsic questions which are of special or unique : 
portance to it. It is beyond the scope of this brief summary to list 
all, but some of the major questions, by initiative, are: , 

.... . ^ 

1. New Ca^reej: Pathways - Can job restructuring be applied in private 
firms as a^ conscious policy tool for increased access of disad- 
taged youth to private, unsubsidized employment (and if so, under 
what conditions)?. Can subsidized training relationships between 
youth and skilled workers, on a one-to-one or small group basis 

be effective in increasing private sector employment opportunities 
and skill development? How can career pathways be developed in 
emerging industries/occupations which make youth competitive 
for entry-level jobs? 
<f 

2, Youth Pre-Employment Services - Do pre-employment services 
sncccod in matching disadvantaged youth with job o{>portunUies ^ 
in ;3mall- to medium-sized firms for which the:%would otherwise^ 
lack knowledge or opportunity? Can the existence of a pi^e-empJoy- 
ment service^impact firms' hiring policies in a way that results 
in a net increase in jobs for youth? 



each 
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Local Business Iiitermediarles (Reducing Transactional Costs ) - 
Doe.^> the establishment of a business-planned, business-run inter- 
mediary^ncrease the effectiveness (i.e., increase private sector 
youth opportunities) of such traditional manpower tools as OJT 
and job placement /development services (essentially a test of a 
delivery system)? ^ 

Youth En t rep renc'urship^ - 'Under what conditions can the business goal 
pf profitability be reconciled with the public goal of increased 
youth employment and training? Is subsidy of business develop- 
ment a cost-effective manpow^ s trat egy for (some) youth? What 
are the most effective ways to provide technical assistance and* 
support to such ventures to enhance their stability/profitability 
(e.g., traditional Tft, franchising support by an established 
company, linkage to commuQity-based economic development or manpower 
programs)? 

Direct Incentives - What level of incentive is necessary to elicit 
significant business **take-up"? What kinds of firms, respond? 
\^at net increase in youth employment among those firms is 
achieved? Do direct incentives also constitute an incentive 
for firms to **cycle*' youth as the subsidy expires? Do windfall 
profits result? Are adult workers or non-*eligible youth dis- 
placed in these firms? 



3 . Design and Analysis Methods 

In all its knowledge development research efforts, CPPV expects to 
combine quantitative analysis with process analysis. The former will rely 
heavily on computerized information provided by CPPV-designed reporting 
forms which will collect information tailored to each initiative, on: 
demographic character^irfil^^s of all youth (including prior employment and 
training history); program^ reports on all youth (attendance, skill increases, 
job placements, promotions, etc.); termination data (exit interviews on 
destination plus nature of termination, reason for termination); charac- 
teristics of participating firms (nature of business, size, demographics 
of work for*ce, etc.); and prograrti costs. Such information will be supple- 
mented as appropriate per each initiative and its individual research 
design by data on firms over time, follow-up post-pjogram data on youth, 
and data on comparison youth collected by CPPV research subcontractors. 

» This data base will be analyzed to assess economic impact of program 
on you/:h, and impact, as relevant in various initiatives, on the hiring 
pracrices firms. Typically, multivariate analysis will be used to 
determine the variance in program output attributable to youth character* 
istics (e.g., sex, age, prior work history); program characteristics 
(e.g., duration, cost); and firm characteristics (e.g., size, industry). 
In ^assessing firm response Lo initiatives, the effort will be to factor 
in both Lhc economic condition of the firm and local labor market condi- 
tions, as availability * of data permits. In initiatives 1-4, the most 
prob&ble comparative measure of impact on youth will be either a simulated 
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control or youth participating in public manpower programs. In the direct 
incentives experiment, our current (and still formative) thinking calls 
fgr a control group of firms, with a youth comparison group formed from 
subsidy-eligible youth seeking employment in control (i.e., non-subsidized) 
firms. 

As a way of validating termination data from programs, and as a way of 
laying the groundwork for longer-term follow-up if preliminary results 
warrant same, follow-sp interviews of participating youth (and, where 
appropriate, matched comparison youth) will be conducted tjiree months after 
termination in each program, • . 

•ThB process analysis design for each program will consist of a series 
of programmatic, implementational and con-textual questions designed foi' each 
initiative. The data base for process analysis will consist of information 
gathered from -sites through periodic observation, semi-structured interviews 
and, as appropriate, survey instruments, as well as statistical descriptions 
of programs from the CPPV information system. Analysis of process findings 
will be largely qualitative and documentary, although in Some cases process 
findings may be amenable, to statistical analysis through us6 of 0-1 varia- 
bles or scaled ratings in explaining program outputs. CPPV be;.ieves the 
process analysis may be particularly valuable in assessing the responses/ 
reactions of private sector employers (from whom collection of quantitative 
data is inherently^dif f icult) , as well as in providing documentation and 
analysis useful for replication, ' * 



A , Timetable of Research Activities 

With the possible exception of the direct incentives experiment, CPPV 
expects all research activities to begin between now and January 1, 1979. 
In all cases, research will be underway by the time of first you.th entrance 
into programs (several of which are now in planning stages). Final 
analysis of each initiative will be due within two years of start date. 
In all cas^s, as well, a "working paper," based on CPPV field reports, 
proc'^ss analysis or program start-up and information system data, will be 
prepared after the first four months of each initiative's operation.^ A 
preliminary research report from subcontractors will be due on each initia- 
tive (covering both process 'findings and, as available, pre],iminary quan- 
titative analysis) after o'tie year. 

5, Known or Anticipated Research Limitations 

Obviously, this research faces all the. endemic i^ues of drawing 
strong empirical/stajfcTistical inferences from non-]^^*f6ratory settings: 
difficulties in assembling or maintaining approp^ate comparison gro^pps;" 
possible instabil/tie? of, organizations and progrJtw&^ndergoing research; 
cooperation of^-pTogram operators; less than omnisGA.enCj researchers, etc. 
It is further /complicated by. the twin effort to a) deyfelop model programs, 
and b) research those models, since the failure f orTTOn-conceptual reasons 



of one or morje programs in the* relatively small program samplesjdn each 
initiative can cripple ^nowledge"'development (wless the failuf^ is "exem- 
plary"!). Knowledge development concerning private sector initiatives 
inevitable involves considerable cooperation from private sector firms and 
individuals, who h^ve no clear incentive (and m^^y have reWcta'nce) to 
cooperate. The hope is, of course, that careful planning, and putting 
research jLn place before programs^,>egin^ can minimize these obstacles. 
Finally, ^policy must ultimately consider the potential macroeconomic effects 
of programming approaches, but ability to generalize from the micro world 
of the demonstration to the macro leyel is inherently' limitea* 



6. > J)ther Possible Limitatiohs 

In general, this demonstration and-.its cls^pstituent initiative's occur 
in a fluid social, apolitical and econ(5'mic context where any number of factors 
exogenous to tbe demonstration per se can influence *CPPV* s ability to 
mount effective programs — and hence to dev^op krcowledge. For instance, 
the imminence ^af CETA Title VII complicates che 'possibility of testing 
local business intermediariesv $ince private industry councils qiay co-opt 
the ground; the possibility of a Targeted Employment Tax Credit aimed 
partially at youth, as the administration has proposed, may render a direct 
incentives experiment redundant; the political issues which surround afid^ 
and all private <*sector initiatives — but especially direct incentives to 
the private sector — could bias 

7 . Likely Research Findings 

Since we are at the front end of Nt;his demonstration, it is sipiply 
impossible to predict the likely directi^q of research findings in any I 
initiative. . . . v 

8. Implications for Youth Policy Priorities ^ , 

Whatever the natOre and direction of research findings in this demon- 
stration, knowledge development should reduce the uncertainty surrounding* 
sucl^key policy issues as: the cost effectiveness of private sector-targeted 
manpower programming for disadvantaged youth; the most promising approaches 
to such programming; the alAlity of government to increase the demand for 
disadvantaged youth in the private sector through manpower p^grams; the 
level^of subsidy required to do so, and the effectiveness of pure subsidy 
approaches; the extent to which* increased demand for disadvantaged youth 
substitutes fo,r demand for*!'other workers; and th£ characteristics of youth 
for whom various private sector initiatives are mo^t effective^in producing 
increased employment opportmiity , stability and wages. * 

Additionally, the" careful testing and documentation of program" approach 
should provide models for replicat;ion of successful efforts elsewhere in, 
a field where tittle .is known about what can , be , at tempted^ much**less what 
works. ^In addition to Federal, state and local manbower *o£f icials, the 9 
broad audience for ^uch information includes privat-e sector -organizations. 
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Specific audiences which are likely to find information yielded by this . 
demonstration useful include privat.e inductry councils formed under the 
proposed CETA Title VII, and other efforts of Office of Youth Programs, 
such as the Corporation .for You,th Enterprises (which will be, launching a 
broader test* of youth ent'repreneursh-ip) , ' ' ^ . * 

* * 
% • 

9, Likely Implications for Future-Research 

/ Almost inevitably, any ".research effort spawns issues for future re- 
search, what-^ver the findings^ and so one expects the current effort will 
do so also. ' > ' ^ 

. The most import'nnt contr ibuti^bn of Ll;is ^demonstration to future re- 
search efforts is, lipwe'ver,^ likely to be what is ledrned about efforts to 
resea9eT» private sector behavior in response to mcinpower programming and 
policies'. T-hiff is a relatively unusual r'es^earch .e'mpasis, yet clearly an 
important oAe- i f publi/pollcy 'aimed at increasing private sector employ- 
ment of the disadv'aataged is to proceed on a basis of reliable knowledge. 
,The multi-faceted, mtilti-part nature of the research task for this demon- 
stration is a complicating factor^ in some respects, but it does provide 
a rich field for lear*ning more about the feasibility of data collection 
in th^ private sector, ^t-he source^ available for suc^collection and the 
approaches to data collection .which are most tolerable to the private -sector 
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\. Overall Purpose . 

To demonstrate and evaluate the capacity of a local community to implem^t 
""this mo(3el of a voluntary, ccmmunity-based youth service program. 

IK Specific Questions/Knowledge 

1. Ttest the demand from and interest of all yDung people in youth community ^ 
service. 

2. Tfest the interest in y^uth ccmmunity service of all potentially eligible 
types of sponsors (public agency.es; private non-profit organizations; 
cormunity groups and groips of youth) . 

3. Test the capacity of all interested sponsors (especially caimunity groups 
and groups of young people themselves) to develcp meaningful, non- traditional 
cormunity service projects, consistent with a representative distribution 
among the eligible t^pes of sponsors. 

4. Tfest the relationship between the cormunity service, experience and the 
educational process. 

^5..']test the ccrnmunity's capacity to mobilize itself in suf^rt of its youth, 
and secure the maximum comiunity mobilizatican in order to: demonstrate 
the community's commitment and loyalty to its young people; demonstrate 
the cormunity' s esteem for volunteer^ performing service; enhance the vblun- 
teer's pride in their ccmmunity service and alleviate their alienation ^ 
from their ccmmunity; increase the of^rtunities available^to you|hwhen the 
program ends; increase .the likelihood of some projects continuing^^fer the ' 
program ends; and increase the community service provided by the limited 
federal funds available under the* program. 

6. Test the effects of youth cormunity service on a diverse gro^p of individuals, 
youth, beneficiaries, sponsors, sipervisors, facilitators and the cormunity- 
at-large. ; . . ' ^ ^ ^ 
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III. ^ Project Design and Analysis Method* 

One evaluation of the YCS/S project focuses upon the process of iirpleipenting 
the denonstration in all sectors of the Syracuse coimunity. This includes 
the administrative and decision making process of the YCS/S board and staff; 
the inter-relationships between ACTION apd YCS/S and the Syracuse community; 
the effects of the political, social and. economic structure of Syracuse and 
Onondaga County on the implementation of the YCS/S project and vice versa; 
and, an analysis of the volunteers and volunteer service projects. Periodic 
reports fron th^e evaluators will enable ACTION and YCS/S to modify features 
of the program's uiplementation as the need arises. And, they will facilitate 
action's design of future adtoQistarative organization and process in a manner 
ccxisistent .with its aim of maximum local responsibility. 

In addition to the inplementation analysis, ACTION has contracted with the 
Urban Institute to design several studies which may be used to ocxnplete the 
evaluation- of the -YCS/S 'demDn^ation. Itiese include: ^ 

1. Descriptive Study; Basic to evaluating and monitoring the YCS are 
descriptive data on volunteers and their assignments, spcxisors and 
their resource caimitments , supervisors and supervision provided, 
YCS finances awJ expenditures, and other such aspects of the YCS. 
Ttiese data can be analyzed independently or enployed as variables ^ 
.thnxigh v^ich to examine effects .under the other studies. IJie ideal 
means. for obtaining these data would be through a management informa- \ 
tigp. system (MIS) for the YCS. ' ^ 

2. Volunteer Impact Study ; Itiis study examines the effects of the ^YCS 
on the volunteers during their tenure with the YCS. It does this 
through carparisons of YCS volunteers with youths in other programs 
and/or yputh& are not in any employroeht program. Hie study col- 
lects data from the volunteers at several points during their service 
tenure. 

3. Volunteer Pollow-Up Study ; This study is an extension of, the Volunteer 

. Inpact Study. It examines the effects vrtiich the* YCS has on volunteers* 
after termination.. Data are collected from volunteers and conparison 
groups by any of several survey technicyes. 

4. Ooinnun ity Im£jact Study: Ttiis .stucJy assesses tlie efliecLs oE the Y.CS f 
on ''those individuals and j!nsj:it,utipns which have had direct wntact 
with the volunteers. Ttiese individuals and institutions include the 
recipients of ^ the volunteers' services, the volunteer si^rvisors 

^ and the spcxisors If it is desired, tijis study can be expanded by ^ 
"gaps substudy." It would augment the mSin study by determinir*g 
v^ether or not the YCS has acted to narrow the gaps between disadvan- 
taged recipients (e.g., poor) and grou^ of the irore advantaged (e.g., 
non-poor)*. 

5. Volunteer Activity Survey; De^nding.on the type of data which the 
MIS amsses^on volunteer placements, this study can assunfe several 
configurations. In general, it is intended to ^ument what the' volun- 
teersVre doing, and to classify volunteer tasks so that they can bfe 
used ai independent variables in examining YCS effects. > 
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6* Moderator Variable Study : This &tidy is designed to identify policy 
relevant and manipulable project variables vrtiich might affect the impact 
of the YCS. Where possible, the study will examine the effects of these 
variables on YCS impact. A primary focus of this study vould be to 
examine the influence of sponsor and project organizational and contex- 
tual v^iable on the effectiveness of the VCS. 

7. Cost-Effectiveness Study : Nd additional data are collected for this study. 
Rather,^ cost-effectiveness ratios for the YCS are established yia cost 
-data obtained through the Descriptive Study, and effectiveness data collected 
under the Volunteer Impact and Follow-Ut) Studies and the Conmunity Impact 
Study. ' ^ 

IV. ^ Research Timetable 

1. Implementation Analysis 

- Octc±>er, 1978, Start-Up Phase Iteport: 



YCS implonentation activities up 'to the Pilot group and intitial percep- 
tions of the "second wave" of volunteers and sponsors. Initial ocmparison 
of the policies of the prpgram as set by ACTION with their actual imple- 
mentation. 

-March, 1979, Growth Phase Iteport: 

YCS implonentation activities from the start of the "second wave" through 
December, 1978. A detailed analysis of the second and succeedirg groups 
of volunteers as ccmpareid to the pilot group. " <[he evolution of management 
"style" and procedures, the solidification of staff job descriptions, and 
responsibilities, key contextual influence during this period, critical 
^policy d^isions and their effectfe, and descriptions of field operations. 
' Ihe primary entasis is sponsor cfevelopment and volunteer recruitment 
activities. MIS information for this arid the "Start7ip Phase." 

-July, 1979, ^ Maintenance v^^^se Import: 

YCS 'implementation activities during the initial phase of project main- 
tenance: i.e. that period in time in which YCS admin istrative^resources 
beccme totally devoted to maintaining existing volunteer opportunities 
and insuring a ^roductiveVand meaningful volunteer service year for 
volunteers, sponsors and the ccmnunity. ExtensNo "site hisObry" t^porta 
on » activities to date fot 10 to 15 selected projects. An analysis of tbe 
staff transition from a projetet growth to maintenance mode o£ operations. 

ch 1980, Final Report: J ' k / ' 

dscussions from the three previous reports with appropriate updates of 
field implementation activities in the "site history" project sites and 
key , policy and administrative decisions between ^ril and June, 1979. 
A full discussion* of 'the Syracuse political culture and its unique 
impacts on the implementation of the YCS program. A full explication of 
key findings for replication purposes wijth particular' attention to the 
implications of these findings for a National Youth Service. % 
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-Periodic repDrt on the selection of staff for orientation and develc^ent, 
on the recruitment process, on the process of sponsor development and the 
involvement of YCS staff in .pfoject develc^ent, on the types of tasks and 
activities that volunteers are performing, the, service-le^irg agree- 
ments and activities of the volunteers, and educational institutions, 
ana on the project supervisors' assessments of the YCS program, 

2. The timetable for implementation of the studies being designed fcy the 
Urban Institute will be established on the basis of the i)rDpoi^ed designs. 

V/VI. Limitations 

1. It is inlikely that data on the youth volunteers' attitudes toward vol un- 
teerian, .ccmmunity service, and other relevant factors can be obtained 
prior to their entry into the YCS/S program, lb fill this vacuun, the 
designs for attitudinal studiej^are likely to employ comparison groups of 
youth not engaged in YCS/S. 

2, It is likely that the time involved in undertaking prograninatiig and 
research objectives of developing and evaluating the local capacity » , 
to operate this model of youth community service will curtail the eval- 
uation of the ^'saturatic«i effects" of YCS/S - i.e., the effects of 
saturating the Icxal youth enployment macket with a sufficient number 

of service opportunities to provide all available 16-21 year olds with 
an opportunity to serve. . ^ ^ ^ , 

VII/VIII. Possible findings/Policy Implications 

The following are indeed tentative ideas on possible findings and policy 
implicaions based on the limited experience to date. ^ 

1. It rfecjjires a considerable period of time and education for a local 
ccmmunity to understand and ij^iplanent a program irjvolvirg youth which 
follows neither the on^-the-jcb training approach nor the income transfer 
approach to, youth unonployment problans. 'This implies that this model 
o^y YCS could not be run nationally as a "no-strings attached, revenue 
sharing" prd^ran, but may require a gran^ applicatic«i and seme degree 
of federal approval of volunteer activities and projects (including 

the identification of supervisors) . 

2. The choice of service opportunities Available to volunteers, the process 
of individualized match-making between volunteers and supervisors,^ and 
the quality of the service projects available has created a favorable 
impression of YCS/S^amoi^ the volunteers, .gcme volunteers will requiz^e 
a substantial aiKOunt of direction in making a match with a project. 

Also project development needs to be coordinated with the level of skij^s 
.amdrg the volunteers being recruited, especially those with inadequate 
basic skills. ' ' 

3. SEhe^ quality of supervision given the volunteer, is likely to be^a critic^ 
^factor in the volunteer's service experience. The studies should atten^^t 
'to define *the personal characteristics of^^a good supervisor as veil as^ 

the characteristics of an otganization conducive' to good supervision. 
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IX. Research Priorities 

The fiDllowing three areas "^re of general interest to those engaged in youth 

'policy/ The research issues surroundirg national youth service are not 
included • 5^ 

1. Identification and searching examination of succesful youth projects. 
What is "success" and why was it achieved? How'can accomplishments 
of youth be conveyed to the adult population? 

2. What are the expectations of both yoyth and adult sector^ of the popu- 
lation about the constructive roles vhich youth can pla^ in this society? 
How can the generalized feeling about youth as a "problem", be turned 
around? * ' 9 

3. Identification of the personal attributes and motivations of effective 
supervisors of youth projects. Idaitification of the types of admini- 
strative and organizational structui*4 viiich are conducive^ to. effective 
supervision of youth projects. The development of programs to train ^ 
and support "non-professional" supervisors of working youth ^ 
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SERVICE MIX ALTERNATIVES DEMONSTRATION PROJECT 



David Gottlieb 



To demonstrate, the relative effectiveness and the 
/longer term benefits derived from three service 
/ mix models designed to increase the employabilit^y 
^ this* youthful target population. ' i 

The major goal is to learn which, if any, of the 
three mix models generates greater benefits in 
terms of costs; attitudinal "and behavioral change; 
employability and employment retentiofi. 

In controlling for age, srex, ethnicity, employ- 
ability, and pre-program work experience 'will 
significant outcome variations occur among and 
between enrollees in each of the following service 
^ mix models : 

A: Labor In4:ensi^^e ; 35 hours per week of on-site 
job training and work experience*. 



of 



/ 



/ 



Supportive Employment: 3 5 hours per week of a 
combination of classroom type occupational 
training, remedial education, and guidance/ 
counseling. , V 

Supportive Service ; 3 5 hours per week,,, 
a combination of on-site work experience and 
supportive services (remedial education , and 
guidance/counseling) . * - . 



Three sites will* be utilized for this demonstration 
project: Broward County, Florida Oakland, . . 

California, and Los Ange.les, California. A total ^ 
o^ 9CtO youth will participate in this demons^tK^tion 
project 300 in each of tjie three si/tes.-* 

Assignment to the ^7arious traii^ing models* will be 
based upon' random assignment of parti.cipants matched 
by ag^e,'sex, ethnicity, and academic stafeu^r Appli- 
cants will- be informed prior to a^ssignment that ->they 
are to be involved in an experimental/demonstration 
^ro^ram and^ wilL be^required pai;ticipate in a 
series of -test/evaluation efforts.. Every effort wil 
b^ made to^ assure commonality in tKe attributes and 
characterLstic^ of pax:ticipants assigned to each of . 
tTie, pirogram mixes. . . ■ 
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Multi-variate analysis will be utilized' noting 

relationships between participants background 
character|.stics (See II above) .and the. various 
dependent variables utilized in the evaluation 
effort. 

Total project duration wij.1 be 24 months alT 

project sites will follow the same research/ 
evaluation time schedule and will utilize the same 
pre-post instruments. Among instruments to be 
administered are the following : 

Vocational Attitude Scale * 
Job Knowledge ^ ^ ^ 

Job Holding Skills ' 
Work Relevant Attitudes Inventory 
Uob Seeking Skills 

Sex ^t^reotype of Adult Occupations 

Self'Esteem 

Academic Proficiency" 

Test/Evaluation/Data Collection will occur at the 
following time periods: 

^a. Determination of eligibility ' selection to program. 

b. Assignment for program' participation . 

c. Mid-point of program (26 weeks) . 

d. Program termination. 

e. Follow-up post program: 

— 30 days ^ • ' 

— 90 days 
. "'240 days ^ 

Lacl'^s^pf unified control of selection/assignment process 

Lack of unified control in administration of test/ 
evaluation materials. • , 

Lack of available instruments , which are adequate in 
accounting for variations, in aq.e, academic prof iciency 
ethnic/cultural background. 

Lack of instrumentation which would tell ufe something . 




•o 




V 




■ ■ V 

There are a'lways unanticipated events and gaps 
between design expectations and realities. Every 
effort possible is being made in or%er to enhance 
the quality and integrity of the * evaluative effort. 

None — far too early. 

To learn whether or not program mix does make a 
difference. Basically, to show whether or not work 
experience alone provides benefits for low income 
youth equal to^ or superior to other service mixes. 

(1) More care/time in thinking out conceptualization/ 
design. 

(2) ' More pre-project effort in the selection of 
test/evaluateve instruments. 

(3) Greater understanding of reasons ^or the too 
frequent conflicts between program operators and 
research/evaluation people. ' 




- 173 - 



187 



An Economic Analysis of the Patterns ^and 
Trends in Youth Unemployment . 

Richard B. Freeman and David A. Wise 
(Co-Principal Investigators) 
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1. Overall Approach and Purpose • 

Time series ^and cross-section data on individuals and groups are 
being analyzed to determine the charactep^istics of the youth unemployment 
problem in the United States and the causes of the postwar increase in 
youth unemployment rates. We hope to draw conclusions that wil l help ^ 
the formulation of policies directed toward the , employment and unemployment 
problems of youth. The principal goal of th^ research- is to explain the 
factors that underlie trends in youth unemployment and employment and par- 
ticipation rates, particularly of black teenagers. We are analyzing several 
supply and demand forces that can be expecj;,ed to influence* teenage unemploy- 
ment, such as demographic changes in the relative *»numbef of teenagers, 
movement of jobs from the inner cities to suburbs, transitional adjustment 
from full-tijne schooling to full-time work," and the minimum wage, among 
others. We are in the process of determining the ways in which various 
character istips of youth employment and unemployment, such a^tnovement into 
and out of the work force, are associated with the high rates of unemployment 
and of obtaining information on how the /young unemployed finance their con- 
sumption and spend their time. We are analyzing time series data, cross- 
section data, and co*\iort and looigitudinal data to estimate the impact of ^ 
several diverse factors on youth unemployment; rates. An effort will be 
made to identify geographic areas with especially low rates of youth unem- 
ployment to obtain clues as to the factors that might reduce rates elsewhere, 

■ ■ y ■ ■ ^ 

2. ^Questions to be Examined / ' • , 

\ Severai. types of data' are being-used l6 anal/yzeVhc trend and char- 

aAteristics of the youth unemployment problem, revalUateJihe importance of 
virious supply and demand factors in the determiUj^tion of the high and 
increasing rates of unemployment among young workers, and to seek an under- , 
standing of the micro-economic factors that affect the rate'^of unemployment 
of young \;6>rk(»rs. The specific rc^^earch ' ob j ectives ^rc: 

« » - ' • « 

1) To .quantify ' the .ii.ajor chn'rac Ler i s t i cs and trends youth unem- 
ployment, employment, and, parti cipation rates. ^ Such quantification is neces- 
sary to document carefully the nature oi the problem both overall and among 
persons defined by ;various characteristics such as sex, race, and education. 
One i 7c>r^lr-;nt sot .of ij-^'os rrl/itc^s -to differS.ces in. the i ne-'pl oyn-rnt cind^ 
Mpl <»,.::.« nfof young pc^rs^>n5; stt/Ving pnTL-tiiic .iS opposed to full-ti»iie work 
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and to tl^e labor market participation as opposed to unemployment among tfie 
young. Another bas to do with the gross fioj^s or tilTnover in youth unemploy- 
ment, the shoC^^uration of youth uneiuployment, and the extent to which changes 
in turnover patterns may have contributed to trends, 

2) to estimate the quantitative effect of supply-side forces on the 
level and trend in youth unemployment rates using tirafe series, longitudinal, 
and cross-section dat^. The principals supply side factors to be exaimed are: 
labor turnover initiated by persons searching for jobs; attitudes toward low- 
level • secondary' iobs which might lead persons to reject certain types of ^ 
empldyment; family background factors, particulatjly the *role of parents in 
obtaining jobs for teenagers, and of the broken home on black teenage possi- 
bilities; the increased number of young persons on the market due to hi^h 
birth rates in the late 1950' s and early 1960's; and problems In the transi- 
tion /rom school to work.' ^ 

3) To^estimate the quantitative effect of demand-side forces on the 
level and trei\d in youth* employment and unemployment rates. The principal 
demand side factors to be examined are: the industrial and occupational 
composition of the economy, which may have changed adversely against teenagers 
in recent years; the minimum wage and other 'rigidities* in the wage structure, 
which would reduce their'employment relartive to what it otherwise would.be; 

'decline of jobs in inner cities where black teenagers are concentrated; 
substitution of female older workers for young persons, especially on part-time 
jobs, induced by t,he increased participation of women in the labor force; and 
of the potential substitution of older workers for teenagers as a result of 
potential legislation striking down mandatory retirement la^^ 

4) To determine the distinctive factors influencing the unemployment 
and employment rates -of specific groups of young persons, namely blacks and 
women. With rfespect to blacks^ the problem is that the rate of unemployment 
of young blacks has risen relative to that of young whites since the mid If^O's 
and e^arly 1960* s. There l^as befeo^-no serious analysis of this pattern^^s^^et 
and, more generally, relatively little information on fhe s'tatus of black teen- 
agers. One r<eason for the lack of information is the "negative" nature of the* 
unemployinent data: it t^ls us that people aren't employed and are looking 

f-or some work but fails to te'll us wh^ they are in fact doing. /J^rincipal 
goal will be .to relate the increased^^tio of black to. white teenage uiVe^mj^oy- 
ment rates to such factors as the fhift of Job§ to suburbs and to how unemployed 
black teenagers use their time. With respect to women, we want to explain the 
extraordinary increase in the L&bor participation of young women dn -recent 
years and, in particular, n^npoint the^ole of attitudinal changes in this 



pattern. 

5) To examine the decline in unefnployment rates as a, cohort agis, 
thereby casting light on the causes of the high unemployment among the young. 
The key question is whether anything beyond pure age and time underlies the 
cirop in iiiit^nploj :v-nt from 2t)-40 percent aniong 16-19 year olds lo 7-15' pc-rrt nt 
carong perbons in the 20-21 y^ar old age biBj^'et . * If ^ny. f aciur" beyond age 
were found to contribute to the drop 'in rates, we wouJd have an important clue 
to the rsLructural factors behind tbe high unemployment ra^c and possible sug- 
gestions for policy. An import^ant part of this objective ife tro investigate 
the pcs.s^bili t'y of changes in cross-section or cohort/longitudinal unemploy- 
PicnL by a^e,** ' . * 



- 175 - 



ERIc - • - 180 



^ 61 Thfe final objective of the project is to produce two related 
repqrts: lone lengthy technical report . embodying the formal research studies 
undertaken; and a less . technical and shorter report desf^gned to pull the 
findings together to form a comprehensiverpicture. The shorter report will 
.be written, for a broad general audiende and will be intended for wide dis- ^ 
semination. 



3. Project Design and A^^alytic Method 

The research pla^ for*" thisr project involves the work of several 
economists, who ar^ using "^various, dat^ s^ts to answer the questions of concern. 
Most of the work involves detailed statistical analysis (described in Table 1) 
supplemented by discussion^ with personnel officials and others involved in the 
youth labor market/ V/hile each of the analysts is developing his own specific 
mode of examining hiS topic, tl\ere is considerable interplay amon^ the resear- 
chers through frequent group me'etings. A major effort will be made by the 
project directors to distill 'the findings - into a unified picture. 

The use of a variety of data "sets is an important feature of the project. 
The data range from aggregate time series to cross-sectional data for SMSA's 
to longitudinal and cros^-sect'ion data on individuals. While our ultimate con- 
cern is with explaining the trend^and level of unemployment rates among the 
young^ the standard problems witfi'^timfe §exies data makes it incumbent on us 
to conduct our analysis^ with a fiill set: H3f individual and aggregate data. 

Cross-section and individual data sets Can be used to test the effect of 

a, 

variables found to be importan't in the* time series analysis. 

The data sets to be analyzed are shqwn in Table 1. The principal 
characteristic of eacM body of data i&'givfn in column 1. The way in which ^ 
they are being used is set outAin column 2^? , ^ ^ , < 

Interviews with persons) i)ivolved iji^th^*}roujth labor market wi"?^ be 
made at various points in the project. In additi^n» an effort will be made 
to bring 'outride' knowledgeable persons to sojie of. Che Bureai meetings on 
youth unemploy'ni^nt. It is hoped that this will help us to ^ain insights that 
might otherwise be missed and to provide us with checks cfri our interpretation 
of results. We will condu<it our own interview survey of the allocation* of 
time of black unemployed teenagers. ^ The survey will be conducted on a' small, 
scale in Boston. The purpose of the. survey is to gain insights into behavioral 
pa-tterns, •'that we would be likely, to miss through analysis only of 'more ' remote 
data sources. ^ " * * * 

By exarmining the potential causes W the high and increasing rate of 
unemployraeni: of teenagers with 'several different types of data and hy bringing 
the insights and suggestions of nonacademic persons to bear on '?he problem, 
ve anticipate putting together the most cr-plete rind valid picture of the* 
■ oiith labor ^.arkei pfobler that has been attc/r^ted to date. ^ . * • * 

^ The research work will be subdivided ini.D several separ'ate repprts,* 

each* to be conduoted by specific scholars, and distilled, in 'a final report ' 

by the project directors. The principal specific projects are listed below. \ 
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Table 1: Data Sets 



Name 



1. 



2. 



NLS Young Men 14-24 
in 1966, followed 
through 1977 



NLS Ypung Women 
14-24 in 1966, 
followed through 
1977 



Characteristics 

y 

Detailed job history and 
personal characteristics 
of^ individuaj^; longitu- 
dinal file w 



Detailed job historyt>and 
personal characteristics 
of individuals; longitu- 
dinal file 



Use * 

Will be major source of 
data for examining de- 
tailed characteristics 
of unemployed and employed 
youth and for evaluating 
changes in unemployment 
over time fc/r individuals 

Will be major source of 
data for exajj^ing detailed 
charactgj^^i^tics of unem- 
ployed and employed youth 
and fpr evaluating changes 
in unemployment over time 
for individuals 



National Bureau 
time series 
data bank . 



Basic time series data 
on labor foxce partici- 
pation and emp 1 oymeat 
by age . 



Key input into time series 
regression analyses 



Current Population 
Survey, March ^1969- 
1977 . 



Current IJopulation 
Survey, May 1976 



NLS, High 

Class of/i972, an<J 

/ollOWyfipS . . 




In^^rview 



of 



l%Linploy'ed Tecn^^ers 

8.^ CenHljJS of Population^ 
^^^L95^i 196a,-l^f^ r 



Gross^ flows-, data 
from CPS * ' "* » 1 



J *■ 



•Cross-section tape with 
data on annual income 

^ a,nd standard CPS^charac: 
teristics of*workers 



Detailed questions on job 
seairch'and wage ra'te data 

etallld information on 
ooling of young persons^, 
with longitudinal follow-ups 

Small scale project de- 
signet) clrt.Xjrniine ^al]oca- 
tioix of ^^ime pt hlHci^ 
.teen^^rs. 

.iJat*^'^ on yn e mpl oymen t , 
emplqjupent arid other 
;*fac^*63ps by -age^-fpr de- 



Will be -used to examine 
trends in teenage unem- 
ployment^ related to 
characteristics, 'and 
family income 

Permits' analysis of mjthods 
of job search by teenagers » 

4 

Major data source for study*^ 
of school-work: transition 

Provide , insights into job 
market problems of black 
inner ci ty 'teoYiagers * 



Prime 



0 * , * ^ 

Dat:dv>n >npvement in anS 

**out -.ofAwoVk 'forc'e 'by ^f' * 
individuais^p t / ^f'^ 



data source iot* cqm^ 
pajmg teen^-gie, ^unemployment 
jivrates acr|)S^s ,S^iSA' s api ^[ 
ti^ai^^ in a,l-€as wit^ dt'f- 
ff^eilt 't)ve>a^^ ^a^r^elS^. • ^ 

Uiil be'\be|'^4:^o^inve5rag^J^" 
' charaq^ei:istics cff. tjf^nage 
* anl^Dplo\Trfnt/ ^ ^ ^ 
" ^ V- V • . / " 
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. The ^{jroject directors keep in close contact with eaSh of the authors * 
and maintain an ongoing review of the progress of each study. * The periodic 
seminars and confeiences' involxihg all of the participants in the project also 
serve as a primary' vehicle for mutual exchan'ge of ide^s ^nd suggestions for 
the improvement of individual studies as well as the project in general. 



A. Time Table 

V 

November 3-A, 1978 
May 17-18,. 1978 



June, 1979 



Wojqking 'conference*. to discuss preliminary 
re^arch results (papers). ' ' 

Conference to present and discuss final 
research papers- 



Executive Summary of major ' results" 



5. Research Limitations 

Our initial findings suggest to us that the youth unemployment problem 
goes beyond traditional economic analysis. First, the problem is concentrated 
among certain subgrot^ps living "in particular social environjnents : high school 
dropouts, blacks who do not go to college, and^thase from disadvantaged families 
Second,' much ^of the'problem shows up in persons out-of-the-labor force, about 
whom government statistics are; noninf ormative* We lack sufficient sample sizes 
for the groups where the problem is concentrated; have little inf orination on the 
career' progress of dropouts; and.no real knowledge of what persons out-of-the- 
labor force are doing. Since available data sources do not provide needed 
information about the most severely .affected groups, it is difficult to see " 
how* these limitations can be overcome in the short-run. ' * ' 

» 

More broadly , we believe that the interrelations among "various problems 
facing the young broken homes, illegitimate children, welfare, crime, drug' 
abuse — suggests ^that the youth employment problem cannot be treated in isola-; 
tion but must be viewed as part-and-parcel of a broader range of social problems 



Ga 

Thus far, we feel that our f inai^n^roduct will conform in all important 
respects to our initial intentions. \^ * 



— Unemployment r^^teS caTcul^Tted fj*onj longitudinal surveys (NLS of the 
1972 High School^ C1 ass) are considerably lower than those base<3 on 
r (TS flata. , * ' ' • . 

. — *Tl->e October 1972 I'^cTijiloyment rate, of Spriiig 1972 high school - 
*3eniors (based on t;> . NLS of the 1972 .High - School Class), was .102; 
the rate for white : \\es was .05A; and for black malts was .130. 



The hourly wage rates of blacks and whites in five successive 
Octobers, beginning in 1972, were about equal (based on the 1972 NLS) 

■ ^ . - / 

4 

Post high school labor force' experience is not greatly affected by 
high*. school 'Vocational" training. r ^ * • , 

Work experience during high school has a signific-dnt positive asso- 
ciation with H)oth wage rates and weeks worked in the years just 
^fter high school. . . , ' • • 

> Achievement test scores, high school class rank, and parents^ 'income* 
are all ^positively associated with post high school wage rates. 



' There "is _s lgn±fircaht" varfaTToh an youth employrtent /population ratios, 
participation rates and unemployment across geographic areas, indica- 
ting that the probl^em is less (more) sever^^ in some areas than in 
.others. 

- Much* of the difference in youth employment patterns-'by area is 
associated Kith labor demand conditi6ns as reflect-ed in aggregate 

. unemployment rates, 'rates 6f gro\^th of income in the ^rea, and the - 
distriljution ,of Industries. ' . . , 

- The youth emp-loyment problem is intrinsically linked to other social 
problems, ^^ot-employeji young persons fend to ^come from families on 
welfare-, in public' housing projects, receiving food stamps, and from 
families headed by women. ' ' 

- There is some indication that young persons who h^ve an ^initially 
poor employment record suffer for several ^ears from this experience. 

^ The major group suffering from joblessness is young blacks. Black/ 
white differences in empldyment, unemployment, and l^bor participar- 
tion cannot be explained by other factors. Much of bl-ack joble'Ssness 
is aasociated. wi'th low labor participation rates and high unemploymejit 
among 'entrants to the labor market. ^ ^ ^ * 

- A large part, of youth unemployment is acccmnted for by a relatively 
small,' h^rd core group of young people who expei^ence long spells 
of unemployment. Whife.most unemployment spells are short, this 

' is due* to ^ the high rates of labor^ force withdrawal, rather than to 
job finding. Amoog' ipale teenagers out of school, fdr^ example , over 
half "of - unemployiTient was due to those with more- than, six months of 
uhomployment in', ihe^ -year. ' ' * ' 

- Theje' is a closfe link. between the social pathology plaguing young 
* blacks and joblessness. Many not* employed black youngsters are on 

drugs, en-gage in crime; \an'd exhibit oth6-r socially abc-rranjt behavior. 

- Demographic patterns resulting in an ilicrease in tne- nurrfbelr .of young 
persons on the lac-or :^rket have contributed to the increasing "rate " 
of unemployment, and decreased labor participation rates, and has , C^' 
acted to depress empl oyment/populat ion ratios. ^ 
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Policy .Implications 

— Policies should be directed toward a rather small proportion of 

yoyth, such as black high school dropouts. 

^ » 

— Policies must be directed toward bringing these youth, or more 
, * likely future geneT;ations of them, ' into the labor force. 

— While traditi6nal aggregate demand policies will have some affect 
on overall youtb unemployment, they should not be expected Xo solve 
the prpblems of those that are the hardest hit — ydung 'bracks. 

— j^We fear that only a broad ba^ed policy aimed at' changin^g the whole 
social environment of the most severely affected groups will sub- 
""^ — ^ — sranfially impTt^eTtheir employwetTr-'pru3;pects. " ^ 



Research Priorities . 

' ' ' * ♦ . 

— A special detailed survey of the groups ' suffering most ^rom youth' 
joblessness with particular concern to out of the labor force young 

persons. " ' • 

*• , < • , ^ 

— Additional survey questions. to obtain information on what the not- 
empl9y£d yotog persons do, bow they finance consumptipn, how they 
eventually move into permanent jobs. 

* — ; Research* shVuld focus more specifically ojv.the groups with the 

wor^t employment problems, and should deal with the full specttum 
of social, ills facing tliese groups, ,not just traditional ecpnomic 
concern^. • ' i . * 



— The youth emp;Loyment problem should be related to personnel policies- 
of emplpyefs. ' , 



The Potential Impact" or^^ployruent and 
Training Programs on Youth Unemployment/ 

(Richard Toikka) * 
( 

Teenage unemployment has been at high 1 vels in the United States !^n 
rec ,int years^ Explanations for this fact- vary. Some contend thaf ' tHe 
measured rates are misleading because full-time students are included^^. 

Others sug ges t tfiat mini mum wage l^ya curtflil dpman<! for fppnagp l^hnr^ 

Sti.llf others point to high rates/of voTLuntary turnover among yoiing people 
who "job shop" early in their career . 

Concern *about the high unemployment rates of teepagers, particularly 
blacks in urban areas, has prompted the* passage of the Youth Employment 
Demonstration Projects Act of ^377. Thisf laW creates new youth programs 
and expands funding for existing e-mployment dnd training progr^^^ serving 
* yout^. - ' 

This paper addresses the question of what impact, this expansion of' 
e::aployment and trailing program for youth is likely tp have on employment 
and unerjployment of young people. The method used tfen the analysis will be ^ 
> to specify^a model of .the youth labor market which focuses on turnover flows 
between employment, unemployment, and school and, then, to introduce eiapXoy^ 
nent and training programs and determine their impact on employment and 
unenploymftnt ooth In the shbrt-run and the lon^nin. The turnover nodel 
wj ) 1 ]>i* r.lmlnn'io tho Markov model disctjrjriVd in Tai kka\( 19 76) . * 

^ 'lli'^ I heofi^l l-^jral nodel will .identify those pro^jram cha/ncterist lc8 which 



i.._ara impoct'mt in^-af f ectlog-Cjnploynient and ifhemployment^. The following pro-* 

gram characterisjtics will be cvalyate'd in the modal: (1) the extent to which 
' the program is' targeted at a particular group, e-g., the un'omployod, low 
^ "^income, etc*; (Z)' the placement rates '6f individuals 4'eaving the program;* ' 



O) the scale of the program;* (4) theVtiming of the program; and (5) whether 
or not the program changes the participants longer terra labor market Success*. 

' Inferences vdll be drawn about the impacts of previous emplojrment and 
training programs on youth unemployment. This analysis will be similar in 
objective to the analysis conducted by Small (19^2); however,' the data* will 
be Interpreted in light of the theoretical model ^described above. Finally, 



)e anjte 



based on present knowledge of the parameters of the labor xaarket model and 

the program characteristics of the expanded youth programs,^ projections will 

♦ 

» be ^de of the impact of these programs' on youth^/Unemploym^nt in 1978-9. 
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' The Social and .Economic Significance 
of Teenage Onemployment 

(Richard Toikka and Frank Levy) • 



I n^ recent years, both the public at large and' governraent policy makers 
have shown increasing concern over high teenage unemployment rates. Evend 
during th^ low unemployyient years of 1967-69, the unemployment rate for ?indi- 
viduals aged 16-19 never fell below 12 percent. As the economy entered the 
downturn of 1974-:75, the 16-19 year-old unemployment rate rose above 19 per- 
cent and tVke unepplbyment rate for certain s ub^ roup s ~ e . g . 7"^^clr m al e s aged 

o 

16-17 — exceeded 38 percent. In reliction to these figures, Congress passed 
the Youth Employment and Demonstration Projects Act of 1977 which created a 
young adult conservation^corps, a program of community improvement projects, 

\ ' ' ^v. ^ ^ ^ ^ I 

and k variety of other public job programs for both In-school and out of 
. schoo^R^ youth.. ^ J , 

Despite 'this, concern, the feeling remains that the problem of teenage 
uuempl'oyTn^nt is not yet in perspective. How,, for example, should one compare 
a teenage unemployment rate of 17 percent to a head-of^household unemployment 

/ 3 f 

rate of 7 percent? Uncertainty over this question exists in two areas* The 
first concerns the extent to which teenage unemployment reflects true hard- 

ship in the short run — the inability of a household to o]5tain necessary in- 

> 

come. The Second area involves a longer nm concern; the extent to which a 
teenager's unemployment is the first signal tha.t he or she will experience 
contin-ued ^abof mark^e^t problems over his or her career. 

V/itJy respect to^ short-run hardship, sone observers a'tgue that teenage 
unemployment rates seriously overstate the amount of harm involved. In their 
view> teenage unemployment involvos a. good deal of job shopping and voluntary 
turhoyer as teenagers- lower unroali stically higli expectations. In nany cases 
teenagers can afford this leisurely search process bccnuse th(5y are raipported 
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# by their parents — ;their. income i^-not crucial, to the household* Harwobd 
(1969) cites anecdotal. evidence to. indicate .that high reservation vages and 
prolonged search create voluntary unemployment for teenaCgers from low incone 
households as well* * * ' ^ * 

Conversely, other observ§rs argue that teenage unemployment rates under- 
state the short-run harm involved because they fktl^ c^unt high numbers of 
teenagers who need and vant jobs but who become discouraged a^d drop out of 
the labor force [Thurow (1977), Levitan and.Taggart (19.74)]. -In the fjlrst 
quarters of 1976, the Current Population Survey (CPS) tabulated an average of 
1»6 million persons aged 16-19 who said they wanted "a jpb but who for >fariou8 
reasotis were not working- or looking for work.^ This 1.6 million figure ^ 

* 'equaled about 20 percent of the nutnber of 16-19 year olds who 'were in^i^e' 

labor fqrce at the time. s^cot^d^understa'tement of harm may^ccur because 
unemployment rates fail to capture teenagera who are working part time but^ 
who desir^ tk) work full time* The relative magnitudes of t^^^ overstate- 

y 

inents* and understatements are unclear. ♦ * , , . 

The long-run implications of a teenager's lineraploytaenf ajre equally un- 
clear. Somd observers argue against any long-^n lii\pact ^r "scarring by 
pointing to the way in which group unemployment rates decline with age. For 

* - ^ # ' " 

ex-ample, i-n 1976 all 16 and 17 year olds in the labor force had 'Sti bneifiploy- 
ment rate of 21 percent, all 18 and 19 year olds* had ap unemployment^ rate of 
17 percent, 20 to 24 year olds had an unemployment rate of 12 ptercent and so 

on . . ■ ' . * » 

\ rBut the meaning of these- declining rates is ambiguou^^ because, while . 

older cohorts have lower unemployment rates, they also have higher labor 



1. See Bureau of Labor Statistics, Employment and Earninj^s , April 1976, 
Table • , • 
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forc^ participation an4 greater absolute numbers of unemployed* In 1976, 
an average of ^8 miH.ion 16 and 17 year olds were unemployed' each month as 

' r ' ' • . 

wer^ .9 million 18. and -19 year olds aind 1» 7. million 20 to 24 year olds* Thus 
it iS/ poajsible that many of those who were first unemployed when they were 
16 and 17 could continue to have serious labor market difficulties in later 
years .even the unemployment rate ^or their cohor.ta declined* 

To summarize, the current problems in interpreting teenage unemployment 
rates center on three ^questions: ^ , 

a) To what extent does teenage 'uVierac^lrQiymen impose an economic 

' ' hardship on the teenager's family? ♦ • * ' 

b) To what extent does the teenage unemployment rate arise * 
from voluntary supply decisions? \ 

c) To what extent dpes unemployment as a teenager presage 
unemployment .problems in subsequent years? 

• * ' t, ' 

f The research discussed in these pages focuses on shortr-run labor market 

decisions by^ young people^ ^or data, we are using a matched March/ Apr 11 

Cui^rent Population Survey which has' i^nforraation on individual and family ^ 

characteristics and can be used to estimate movement between labor marked 

states such as employment, unemployment, .and noa^-partic ipation in the labor 

* - • . ' 

force. * * * * * . , '* • ' 

The research plan follows a path bto^en by a number of xesearchers 
LHdlt (197^)j_Toikka (1976); Smith (1977); l^iacston (19760; Barron and-Mellow 
(1978)] in that the shprt-ru^ (monthly) chang'es fn labor market stAte are 
Tifbdelled using *tran3ition:probabilities. The three ^a te ^emplo^ent/ 
unemployment /not- in- the. labor force) M.irkov model used by ra'any^ analysts will 
be extended in directions which are appr/5pr l.^te f or*^(^udying the behavior of 
yoMt^- Six labor jnarket states will be of iiitere.st: (1> full-£ine employ- 
ment, (2) part-time cmployir^ent, (3) une nplo/inent of those wlio lost thei.r 



laQ^ Job, (A) unemployment of those who Ipft tli^^r last job, (5) uneinploy- ^ 
went of entrants 'or. re-entrants, and (6) non~participation in the labor 
force. . • ^ * ^ ^ • 

.The first stage. itv the empirical research invdlves estimating the pro- 
* , * « * «, ♦ ^ , 

^ability of being in each of 'the six labor market -staple's as of March 1976 
conditiional on a series of variables which /will include age,.race> sex, 

, ♦ c • ' ' 

education, family income, "sex of family head, presence 'of j^oung children,^ 
previous year's labor force sta^tus, area im'employmeat- rate, degree Of 
urbanisation, and* all Imputed wage/ These tabulations should provide a. 
sumriiary statement of how young people -are distributed in the labor-market. 
This distribution will be analyzed v^th the 'following qu^tions in mind: 
How important is age relative to other factors in determining labor force ^ 
status? VJhat generalizations can be ma3e about the relation between labor 
force status and family income?. To what extent is teenage unemployment a 
•oroblem of urban areas and to "what extent is it*ff problem for non-urb^tn , 



.areas as well. 
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The second stage of the empirical research will entail estimating models 
with transition probabilities as dependent variables* At this 3'tage,"it will 
be easier, to i^ntify the behaviors^l patterns which produced the observed , 
distribution ojf "persons by labor foroe status. -For ^example, it will be 
possible to "determine whether a high (low) rate of non-par.f icipation result's 
frorara high (low) labor force ^it rate or a low (high) labor force jsntrance 
rate. In 'sorae^ caWs, .it will be possible to^go further- aad test interesting 
hypotheses about the effects of age and family income 'on labor *force parti^ 
cipation^ quits and* ISypffs*.- * . • ^ * /* ^ 

After the prdbability functions have-been estimated, i^ will be pos-- 
sible Co characterize the normal labor market experiences of young people 



eharact 



with* certalin ^ypes of 



icterlstlcs* These empirical models cal) offer 



subst^an.tlil Insight inter th6 three (questions posed at the beginning of J tl^s 
^summary. First, because the models Include^famlly Income as an Independent 

- ■ ' ' . •■ 1 ... "-^ 

variable^ they will permit an Investigation of t?l\e* extent to which teenage 

• ^ 4 • • • 9 * * ^ j / 

unemployment produces short-run hardship. Second, by differentiating between 

\ • •. ^- • . 

unemployment arising from quits, the role of voluntary decisions in deter- 
mining t^eenage unemployment can be investigated. Finally, by Includlpg the 
previous year's labor market history as independent variables, the extent to 
which early unemployment experiences signal subsequent labor market diffl- 
'cultifes can also be invesflgated . ♦ 
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» mSUREMENT/OF WORK. OUTPUT "of YOUIH PROGRAMS - DAVID ZIMMERMAN AND ROBERT EGAN 

■ . . • \ -<t- . - MPR , 



PURPOSE AND OBJECTIVES 
. *« " . ' • * . 

. . . - Federal government, welfare and labor pblicy has recently p^laced 

^l(iorfe emphasis on both public service eraployment and employment and 

training^- opportunities -for young adult^. One important objective 

• of public employment progreuns, including those specifically targeted 

toward youn'g-peop^l!^^ i's the provisioh of goods and services. Although 

* - , \ - ' • \ 

providing these goods. £md services is not the primary rapison for implementing 
the prpgramSr differing assessments* of tkhe vaJLcr^ of the output produced 
will undoubtedly have '^an important role in the continuing political i-arid 
academic debate over them, • ^ - ^ 

Tl\e two 'major objectives of thfe research are (1) to provide pre- 
liminary information on the value of output of a samole of yqijth employment 

- .-^ ■ \ ' V- - ■ • • ' 

and work experience projects, and (2) to assess the feasibility and 

relative desirai^il^ty of various strategies for, valuing the out{)\lt o6 j,fouth 

projects in the future 5, ' 

To accomplish these objectives,' a number 0^ specific questions must 

Jae answered; ♦ / ♦ ' * 

(i) From a policy perspective^ what .informatioir^pn ' ^ 

the'^alue of output and .other ^aspects of youth ' ^' * 
employmerit^ projects should be provided? Tlias'is 
partly , a theoretical question based on standa^^d ' . • 
evaluative criteria^' and partly a prfiictical 



questlort^f what information wi^ be most useful ^ ' 

to policymakers and program operators. ' ' ' 

(2) In assessing value^ of output, what are the methods- 

^. for 'measuring the relevant 'variables? Can tftjey be ' J 

^ measured at all? How accurately? How much confi- . % 

J dence.can be placed <in the measure^ When "rnoryB^ ' - , 

" , • than one strategy ^xists, what are ^he advantages ^ # 

and ^sadvantages of each? " \ 

/ (3^) What are the best^inethods for eliciting arid analyzing * 
descriptive information on the. projects? Do the 
methods vary by type of 'project? What arc^^the hQi<;t 5 * . 
. sources of this information? 
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^, . (4) How costly is the measur6ment , data collection, 
and analysj^ likely to be? How does this vary 
with the extent of inforrnation available, the 
strategies adopted, the characteriistics of the 
progx:axp, and other- factors? . . • 

( 

- . \ ' • / 

These questions are obviously interdependent, and'they^ust be 
addressed" accordingly. The purpose of ^th^^tudy is to respond to them 
in a inannjer* that best ineet^ both the short-run and long-run objectives 
by conducting a set of, pilot studies of existing youth employment 
projects. These stud^^es c;aA provide both quamtifiable and descriptive 
project- infortftation on a preliminary ba?is without haying; t*' wait for . 
future research efforts. In addition, actually undertaking a series 
of studies should add significantly to our knovJledge of iiie benefits 
and costs of*^ S^rticuleir project definitipn and measurement strategy 



alternatives. 



PRIME SPONSOR/PROJECT SAMPLE 



In order to insure tha^^e have a good representation of various ' 
^types of projects,- we have stratified the sample of projects along certain 
dimensions • A key stratification is by type of program. The Department 
of Labor has asked us to include the' following types of youth work projects 
in the study: ' * ^ 

~ ■ \ ■ . •■ ■ 

A) Existing Title x Youth Work Experience Programs 

♦ * # 

!• CETA in-school^'projects 

2. CETA out-of-school projects ' 

3^ CETA summ^er work projects 

B) Existing Work Projects of ^fie Title III SPEDY Summer Program 
Cl Programs Funded Under 'YEDPA 

1* Youth Adult Corji:ervation Corp:> (YACC) 

2. Youth Community Conservation and Improvement Prpjccts (YCCIP) 

3. ' Youth Employment and Training Proyreuns (YETP) ' 



D) Existing Work Projects Funded Under Titles II and VI, < • 
in selecting the prime sponsors arid YACC^ sites to be visited, we 



h^ive attempted to obtain a representative .sample in terms of the type of 
prime sponsor (i.e., city,, county, Balant.e-of -State, and consortium) , ^ 
geographic area, and, pd^ulation of tho i»rca. Our plan has been tQ select ^ 
between 40 and 45 projects at 14 sites to be included in the study. Of 
the. 14 sites, 12 are CETA prime sponsors cind two aire YACC'^sites. The 
sample size assumes that we will study Uirec randomly selected projects 
at each site. At some of t}ie sites all three p^rojects studied wiI3 be • 
ybuth projects, while at, other sites we will study two' youth projects v 
and oneadult'pSB project engaged in work similar to that Joeing done in 
one .of the youth projects studies. ^ ^ . 

/field study tasks 

The field study portion of the study i3 structured by the. major 
tasks which we^list below and which follow directly from the objectives 
and issues discussed ciboVev As one ceads through the list, it is clear 
that .pach. task has, subsequent activities ai'ssociated with it, marty of which 
are particular to the .circumstances of the specific prime sponsor or. 
project • Nevertheless, this list of the. ma jor *tasks does indicate the 
scope- of the work and level* of effort required in the study. 

. 1. Determine the state of the local pr ime spopsor and othe r 0£eratin£ 

• * ' 

ciqency pr ogram r ec ords . Many part^ of the value of output study will require 

local i/rogram d^ata in the fprm of* local primn sponsor or other agency records- 

An assessment of the adequacjr of these records will^Jbe critical for the 

accurate determination of the feasibility and costs associated v/ith 
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alternative tndasurefnent strategics. Appraisal of the state of the 
.lotfai program data will identify the constraints, which might e;iiminate 
particular measurement strategics.' ^ * 

2. Test and A ssess S?upply Pric^ Est imation Proc ed ure s 
A major task of the study 'will be to test various methods of 
estimating the price an 'alternative supplier would change for the work ^ 
performed by the program participants. While consideration must be 
given to whether one, procedure .can»be used 'universally across projects 
or positions, our experience indicates that the feasibility of 
alternative profcedures will vary by the type of work performed* 

,Our basic plan will be to us.e the pilot studies to test various 
supply price estimation strategies, many of which we have used in previous 
studies, i/1 order to assess their appropriateness for the youth projects. 
V;e will be able to assess, on the basis of th&^pilot studies, whether there 
a>-e certain tvpes of projects which are more (or less) coi^ducive to specific 

^ \ '\ \' ■• ■ - ■ . - . 

-estination strategies ^and whether there are acceptable cost-reducing > 
nodi fixations of thofee, strategies. . - - 

3. Test and Assess Strategies for Obtaining' Information on the 
DerTiand for the Project Work - 

Deriv.in'g estimates of the demand for the work performed is a very 

^ t« mum 

impo'^tant yet particularly thorny issue. A. primary question to be addressed 
in tTje^study is what particular type of information on the demand tor the 
work is desired £md avaiTaole. For example, will it be possible to place 

monetary value of soc|iety*s demand for the oiatput? Can we obtain 
deficripflfesf^ information relating to the demand for the work, such as the 
oh jcc tAves or rationalti for thcf project., whethe^r thfc work wor; performed 
(or planned) prior to^ tVif' projc-ct , wliether therft/ar(> plani; for t;outinulng 
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the work after the prograro ends, whether there was opposution or barriers 
which emerged in the implementation of the project, eithe^f by the ' , 
recipient' or an alternative supplier of the service or product, how 
much publicity was given to the. project, and various. assessments of 
the, usefulness of the project. Beyond this, the form of; the inf orritation must 
be addressed: Can we get quantifiable data on demand for the work, or dq we 
rely on descriptive information? One objective of this pcirt p^the ^tudy 
will be td. determine whether a single set of proceduras can be used for ' 
all assessments of demand, or whether the procedures ^ust vary hy^ type 
of work or some other dimension. , 

" " - . V. . ' ■ . • - • . 

4. Test and Assess Strat^ies for 0)^taining Informatoh on Project 
Costs and Inputs 

llstiatating the costs' pf the work performed is 'extremely important 
in interpreting the value of output results. However, estimating the costs 
of the project, particularly the sociSl costs as opposed to the budget 
expenditures, . is usually notf a simple task-. In many cases the program 
accoun^ng system is not structured in such a way that all th^ costs 
associated with, the particular project or job slots under consideration 
can readily be identified. Costs of adltiinistration, supervision, equip- 
ment, of materialjs, which may be substantial, may not be included in accounting 
data, or ^ their treatment may vary from project to project. The expenditure 
data may not contain all the costs or the correct costs^ needed to calculate 
the social cost of the work. Our experience has shown, in fact, that in 
"inany cases it is the* most: time consuming of all the tasks associated with 
thfe value-of -output effort. In liLght of this problem, it has been 
necessary to test the cost data collectign strategies on a sub-sample basi5>. 

In addition* to assessing the availability of cont data, the «iv.irl 
ability of da^ba on labor input'.* H^^oci.lt^:d with Ih • pW)jrx t m\i'W. .ili.,> >»• 
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assessed. Almost any cor^ceivablc? use of the results of a value of 
output study would require that they be put on a ''per hour", "per month*' , 
••per yeaV", or "per enrollee** basis. Our experience on* other program ^ 
evaluations suggests that this conversion is both important and 



potentially error-prone. This .task, therefore, will include an examina- 
tion of -l>oth national and local data on participant labor inputs, and the 
conversion of the value of [output estimate to a per period of service or 
per enrollee basis will be attempted for every project studied* 

5. Investigate the Qeterminahts of the Value of Output and Cqsts 
Another important part of the project studies will be to obtain 
"infomation on the determinants of the value of the project output^ These 
d-eteminants could range from the overall motivation for the project to 
such' things as rhe eligibility criteria for the participants, the work 
technology and flow, various aspects of project supervision, disciplinary 
^ ^problems r .'.ax^d even the amount of visibility or publicity given to the 
project* ,v?hile it is unlikely that the information elicited Will be 
, sufficient: to permit extensive project-by-project comparisons in this 
area, it shoulli* be possible from the pilot studies to generate some 
hyootheses concerning the- determinants of value of output and to better 
define what types of data oh value of output determinants are readily 
available. * ^ , * ' 

S . Determine the 'Representativeness of the Projects Studied' 
In addition to the individual project studies, information is being 
collected to determine the representativeness of .the. projects studied, 
in terms of the type of work .performed and any other (3imen5:iions wh^ch may 
iie rolcvcint in interpreting the results ot the p^^lot studieii or planning 
future T?ailue of output, research efforts. Ivhilc is highly unlikely that 
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a sophisticated analysis of the representativeness of the studied 
projects can ba undertaken within the limited scope of the study, 
there ohould bfe data available to make at least ^ crude comparison , 
of tlie projects studied to other projects at the sxte, , • 



SCHEDULE 

The study is scheduled to be completed in^ a ten month period.^ 
It began in March, 1978 and is scheduled for completion in January , 
1979. The primary design pha^e v;as completed in March with some 
minor revisions made after the comoletion of the first set of project 
visits in April and May, J.978. All field studies are to be completed 
by early October, with the summary and analysis of the data collected 
to be undertaken during the mbnths of October and November. A draft 
of the final reoort is scheduled for the end of November, 



ERIC 



2ii: 



- 196 - 



' Project Overview Statement 1/ 

A 

- ' ' . . J • " 

Continuous Longitudinal Manpower Survey (CLMS) 
. * ' of YpPA Programs ' 

by Seympur w Brdndwein , 
. ^ Director, Of'fice of Program Evaluation 
•Employment and Training Administration 

The Continuous- Longitjadinal Manpower Survey (CLMS) has 
been tracking an annual national ' sample -of participants' in 
CETA decentralized programs.' It is now being expanded to 
include a sample of enrollees- in the ,two YEDPA formula- 
funded programs: ^Youth Community Conservation and Improve- 
ment Pro jects (YCCIP) and the Youth Employment and Training 
Program (YETP)\ ' ^ ) 

. , This pap'er briefly reviews what the CLMS is, the basic 
pur/os^s of ext(2ndihg it .to YCCIP and YETP, the timetable 
involve{3, some limitations, and several issues in such 
coverage.. * o 

•>> \ 

'' Background on CLMS : 2/ The CLMS selects and obtains 
detailed j^pf o):mation from_ a national sample "^ofi participants 
in CETA /programs .run by State and local governments' ("prime 
sponsors")'. The sampling and data collection are conducted 
by the Census ^ureau, »with technical aid, basic analyses, 
and reports provided , through a contractor (Westat, 'Inc.-)> 
ujider the" direction of ETA's Office of Pr:ogram-. Evaluation • 

The sample is ^designed to provide national estimates; 
it 'does not provide infqrmation on individual prime spon- 
sors. In FY 197.8, the national sample , ^ drawn from 147 
sponsors, was 18,000 new enrollees. This included a sub- 
sample of 2,''000 in youth work experience activities 
(including summer youth wprk programs). Thd remainder 
of the sample is drQ\yn to rcprci^ent the other major CV/VA 
activities, which are (jearod largely to adu-lts (public: 
ser'^yice employment,* classroom Lrainiiuj, jpVi-Lhc^-.jolj trd i nin<|., 
and adult work programs) , but it includes ^(and f)rovjdoi-> 
separate, data foi^ the youth enrolled in such activities. 

" - * 

T7 Prepared for "Knowledge Development Projects Cdnference" 
of Office of Youth Programs, October 5-6; 197^. 

2/ A technical report on. the CLMS, available on request from 
the Office of Program Evaluation, describes more fuily its 
sample „^esign , data collection procedures, and stat*^,tical 
i -confidence levels, and pr^d^nts its data Collection 
instruments. 
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Procedurally, at the end 6^* each quarter, the CLMS 
draws the sample for that quarter in each; of 147 sponsor ( 
areas, checks the sponsor records^ on each enrollee in the 
sample for information on the activity in which enrolled, 
and , then, finds and interviews , the enrollees to get detailed/ 
information .from them*on personal and family otaracteris- 
tics, on^employment, earning, and school exgerrence in the 
year preceding ejitry to the program, and" on their attitudes 
to the prograra* • ' 

About 9 months later,* the CLMS follows up both to 
check sponsor records on the services provided to the sam- 
pled participants and on termination dates of those who*ve 
left the progfam by then, and to interview- the participants 
on their "continuing labor market experience • This has been 
the final interview for" those in the youth .programs , but 
those in the *adul t-or iented programs are followed up again 
at 18 months and at 36 months after enrollment'. . » 

CLMS Covera<^e of YCCIP and YETP ; The same design and 
procedures are now being used to develop information/On a * 
sample of FY 1978 YCCIP and YETP participants/^ ^or FY 
additional questions on youth attitudes are to be added to 
the*survey on an experimen til basis * - . '^v 

The sample will be 3, 000 a year* in tht3 YETP activities 
and'- 1, 000^ in YCCIP; the 3,00.0 in YETP is assJ^ed to be suf- 
ficiently 'large to enable sizable subs^amplos for in-school v 
as against out-of-sch6ol youth and for work experience , 
activities as against class or On-job-i:rainingc ^ 

Purposes of YCCIP - YETP Survey SS The principal initial 
purpose rs of course to d'etermine, in detail not available " 
from the sponsb? summary repor.ting system, just who is being 
served by the two YEDPA programs . The initial interview 
will provide extensive, information on the demog'raphic and . 
economic characte^J^stics of those enrolled* also, will 

permit'^comfferisoii gf the makeifp bf the yout.h .enrollees irj^ 
othier CETA activsities as against thoise. in the-two YE^PA ^ 
programs r A . ,» ^ , . ^' \ 

^ . / 

A second '^rpose is to get the views of participants 

on the prog'rain. The CLMS asks, while they arc still in or 

shortly after they have left the pr6gram/ aifci 'again upon 

followup', ;about their/*"satisfaction" with th^^program, \i\\dt 

they most Liked and (^^liked, and views as to possible ' 

improvement • • ^ * 
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Pi third purpose ,^ from the f ollc3wup tracking , is to 
determine how long fchey s'tay in the program ; whether there 
are notable dis<:inctions dn who-stays longer v^ersus leaving 
early, -reasons^ lor leaving, whether they remain or re--. . 
turn to school, and- what t,heir early postprogram employ- 
ment and earnings experiences are . ' 7~ 

^ ,The data will al^o enable identification '^of major 
patterns of dif-f er.ences between the two YEDPA progra^^s, the 
"major YETP 'components ,' and other CETA activities with large 
youth enrollments. * = - 

, Timetable ; The initial f ocus . is on dat^a to be available 
for the March 1979 report to Cdngress on YEDPA. 

The data collected fTom; sponsor record3 should be avail- 
.able for the^full !sample of FY 1978 (Snirollees. The data frem 
the personal interviews will be available for those ^enrolled 
through 3rd quarter, FY >978. (Since interviews those . / . 
enrolled 'in the * 4 Llj quarter of FY 1978 will not be completed 
until December 1978, it is unlikely that* processing' and^ ^ ! 
analyse:/ ot that data can be completed in tigie for March 
1979 ^report) . Still, the -data that will be available should 
permit- a comprehensive report on just who wa§ being reached 
and served by the YCCIP .and YETP prbgrams ,in their initial 
year . > . • / ^ : . 

Thereaf tfer , , information from the first (baseline) 
interviews, conducted^ with enrolleefe afte;tr the quarter^in 
which, they enrolled, ^'fehould be available 'approximately 9 
mohths after their, entry quarter (al^out^5-6 months from *the 
end'Of'^'the interviewing period). Information from thfe 
followyp intervie\jis^hould similarly.be available' some 
months *after end T)f the interv^iew-H|4^ (although problems 

of constructiiy merged ^pre-post l^ng'^-tudinal files and of, 
^evelopirfg suitable analytic formats*^ probably will require ^ 
a longer interval .for *the first follQWup analyses) . 

Limitations ; One limitatio^n of the CLMS da^a, as^ , / 
'indicated in thfi notes above on- "timetable,", is its timincj 

there is an inhe^^ent time, l^g before the basic data can be 
-provide*, and moi?e time -is necessary for anyicomplex- 

analyses. % * ^ \ 

Specifically, since the survey actiyity does not start 
bill after bnd of a quarter and* then requires.^, sampling , 
checks of the sponsor records, finding and^ intervl^ing of . 



- 199 



^10 



the enrollees, 'processing .^of the questionnaire forms for 
computer handling/ 'development. 'of d^ta tapes, development 
of tables-, and then .analyses, it is 8' to 9 months/after ' 
the end of: 'the enrollment quarter before basic analyses , 
ordinarily ..^an bd ayailerbl^'. ** . ^ . * 

There are variolus 'technical limitations .of^i^eliability 
and/consisteficy on some data items because of wide diversity 
and some .ef raticness in spofisor recordkeeping practices (on 
recording of termination dates, for example ^. and on identi- . 
fication of activiti.es or services provided), ,but^hese^' 
shoxild not .seriously limit the utility of tbe basic CLMS 
^data. ^ • . ' * ' 

^ • . ^ ' \ 

Issues : Three'major design issifes have arisen which 
warrant .comment . ' • q , 

(1) The Office of .Youth Programs believes 'if would ble 
desirable to develop a "psycholog^ical"^ p^.ofilf of YEDPA 
participaDts arid also to try to,, gauge whether -trertain' ^^ti-^ 
tudes change after program particijiation • It developed 
questions about , perceptions of' work ,v (Occupational aspirations 
perceptioas- of future oppbrtunities, self-est^m, and other 
attitudes-. 'Th.e* questions , are similar to those in the Parnes 
Longitucjinal Surveys or in several other youth ^tu<^ies, , * 
/ ^ • , ^ 

The Office of* Program Eyaluation and the Censrus' 
Bureau were dlabious about adding these qdestitDns a bit 
because ^of .Concerns about utility, but more so because t)f 
boncern about .theix effects' dt\ answe^rs to other .questions 
and on; response rates, as well as concern about bias where 
parents or others are pres'enjt_during the int^:^iew^. 

•The compromise decision. has been to add such questions 
on a^n experimental basis for/the^FY 1979 sample. The ques- ' 
tionor (assuming 0MB approval) vWill be asked of. half the 
enrollee sample, a's^a means -of testing the effects suoh ^ 
questions, getting the information if it proves^'practical 
to do. so reLiably without adverse effects, and reducing ^- 
the risk of adverse effefct on the entire sample .if it turns 
out that the quest^ions .do significantly, alter responses and 
cut 'the response rate. ^ . , . ' . 

. ' • . ' ' ^ ' 

(2) A secbnd issue is the timing of the followup, interview 
and the number of followup inter.views/ At present the ^ 
first CLMS followup interview, inten4ed . to 'provide early 
information on initial postprogram e¥^erience, is approxi- ^ 
mately*9 months after enrollment (about .6 months after .the 
initial interview) , ' This limited period between the 'initial 



and tjjie second interview has probably *helped maintain* 
response rates and e^sed recollection problems. But on ' 
the negative s^de, the, limited. time before the followup 
interview hafe meant that many or -most enrollees are still 
.in the program^When reint^viewed , ' so that initial post- 
program data is- just fox a reduced sample a^^d only for 
relatively earlv\ leavers^ 



Consideration^ is therefore beiijig givea to deferring 

• the f if-st .followup until 12 to 15 months after enrollment, 

to get at least ^ome postprogram informatior\ on virtually^ ^ 

all. the YEDPA enrollees ^ This presents logistic problems- ^ 

^for the Census / however', which will have field scheduling 
problems if different CLM5^ followup ^intervals are used for * ' 
YEDPA ds against other CETA enrollees. A. likely answer is 
that the followup interval may be changed to 12-15 Apnths 

•for all of tji^.^MS. . - * , . ' 

, • As to the number of followup interviews, tHe YEDPA 
enrollees are tentatively ^chedMled 'tb be followed up 
twice (compare(^ ' to -just once'for other ,CET:^youth programs 
and three times for .CETA adult-oriented programs), Th^^__y 
second followup would be ISs^months after enrdllment; if 
the first followup 'is tieferr'ed frorn the prijfi^nb 9 mi>nths^ 

• to a^,nfew 12 or 15 months, it will make sense to def-er the 
second followup to at 'least 24 months after enrollment. 

Whejyher to retain th^ planned second followup or add 
^a third followup will be decided at a later point in light ' 
of experience and findings of the, first followup. 

(3) Finally, there is ±he basic issu^^? of how to estimate 
net impact of.theYCCIP and YETP' programs upon participants. 
Th¥'CLMS .followup data Will present , gross information ^on 
their .postprogram experience, *but will .n*9't be. able to de-ter^ 
itiine how much of any level,, or aoy pre-t*o-post change, is 
attributable "-tO' program participation rather than other • 
' factors. ■ ' ^ * . • ' - 

* . For the, adult-oifiented CfiTA '^!)ro(^rams, the CL'MS as- 
developing a' comparison group from the Current Population 
Survey, and will co^mp^re its SociaT Securi^ty earnings with 
those of ■>the CETA participants. TfYe differences in nature 
and extent of earnii>gs changes for the two groups will> 
indicate the net impact of program participation on partj.- - 

• cipant earnings. • / ' " - ^ ' 
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A Similar effort is not b^mg planned specifically, 
for. youth program patticip^nts beqause the unsettled move- 
ment of youth in an out of. school and in and ,olit of the labor 
force severely limits the meaningf ulness of any earnings 
comparison with a constructed comparison group and requires 
a longitudinal comparison at Ij^st some year^ into adulthood. 

At the moment, general plans are to explore whether / 
youth comparison^ groups developed for other yojath program 
evaluation/ 'most ndtably that o£ the Job^ Corps program 
and of the Supported Work program, 'as well as the CLMS CPS 
youth group, ^might ser^e in coml^ination as sensikjl^ "'brter-' 
qjnce poi'nts" for estimating net effects^of YEDPA on its 
participants. ^ * * 

\ ' . • 




f 

PATHWAYS INTO THE WORLD OF WORK) EJCPERIENCES' OF YOUTH 
THE NATIONAL LONGITUDINAL SURVEY OF YOUTH . 
The Ohio State University 
Princi'pal Investigators - Michatel E, Borus and Herbert S. Parnes 

Overall Purpose ^ * ' 

The Office-of Youth Programs, ETA, United States Department of Labor, has 
tended an extension of the National longitudinal Surveys of Labor Market ^ ^ 

Behavior to cover a new sample of y'outh. Annual interviews will be conducted 
between 1979 and 1985 with 12,000 young men and women, age^ ih to\21. TMrS 
-longitudinal examination of young fjeople, which will pay particular attention 
to those groups with above avei^age Xkbor market di^iculties ,^ seelss to provide 
theCdata to answer maay of the urgent policy wqaestions dealing with the movement 
of youth from the classroom into the work place. Among the i^i^es*" to be " ■ 
studied are ttie effectiveness of Depar^^ment of Le^or employmentund training 
programs, the factors affectiJig a youth's educational Experience ~and educational 
attainment , ^he det^raainants of the Job or Jobs into which the youth will move, 
and the fact^rs-^inf luenci^ng marital, child-bearing, and .familial decision- 
making. . . ^ ' * 

This new study is an extension of the National "Longitudinal Surveys which- 
have been'conducted since I966 by The Ohio State University Center^ for H\imar> 
Resource Research under U.S, Department of Labor contract^ and which. have resulted 
in the publication of I8 research monographs by the Employment and Training 
Administration and oyer 200 articles, theses, monographs and special reports. 
Fot the pa'st det:ade the National Longitudinal Surveys have been one q,f the basic 
data sources for social scientists - economists, sociologists, psychologists and 
educators - who have sought to tackle employrient and training issues. NLS 
data tapes and doc{imentation are av^,ilabl« at over I50 institutions in the 
U,S. It is expected that data generated by the new study will allow researchers 
throughout, the nation to develop solutions ^tp many of the problems which face 
youth in bridging the gap between formal educatior^ and training and the 
establishment of relatively stable attachments to Qiven types of work. 

Basic Relsearch' Question5**to be Kxteininea, ' Y * 

"I'he vrinl .'jiaouiit, c/; Jcjla that wU u tiir. .l/' * prnvi^'ed i^^ Up- ru-w 
N;i.tional I/'>ri/';i tud.lnai ;;iirvr,\ c')!' Yow.-ii n^y ]/[ I'lc-raiJ. n «:)drea:: of 

quefjiion:: to be answer '^i. iu'lov ru:-^ I n^m-, oi* .f^vu r' :.oiirch togi^c:: 

that we propu.,f^ to li'ttuiy. ^\ • ^ * 



Evallaatiori of Dejiarlmen^ of Lauc -r j^^-/:'^-rit i\y\ '-'v'u. unz VTo\i,r'xnz . The * 
Youth Employment and DemonrA rat ion ?rojcctr. Act, of r.'il ^^d^ed several new orogrfims 
to those designea tc3 upgrade the .%ki Li:-> -jf aidiji.' v^'ci, a:ideremployed , and ^ 
economically disadvantaged youth."' It also exv.ir.d -o fundini; for youth programs 
provided by (5)TA prime spunsor*. In all, a aojjrvjxiiiiuwcly 2 million youth are. 
^likely to be enrolled^ annnally. In "W^k^ o\en^..' ^.tiafle we estimate that '^00 .youth 
will participate in one or more employment arnj tralnlnr!; programs, while almost * 
2,500 ybi^g persons wij.1 be eligible but wiiJ not uarticip?ite. ^ We propocje to 
use these groups to conduct two types of evaluations of these programs. 

\ 



Cl) ^he first typf of eva^luation will study t>he process of the programs. 
Participants will be asked their reactions to the program in which they pajticj.- 
pated, how it helped them in the labor piarket, and what facets of the, program 
could be improved. Responses will be ^related to the individuaJ-s • characteristics 
'and backgrounds and. to the specific ^-ervices they received in an effort to discover^ 
.why peoj^le choose to g^tl^icipate in employment and training programs, what 
^-S^rvices are bei^ provided^ how vfell they are received, who drops 'out -of * 
programs , and v^^^^ight be done, to improve the programs'.^ - ' a ' 

(2) The second^type of evaluation will stud;>" the impact of the programs . ^ 
by cofliparing the post-program labor market e^iperiencec of p.a:&ticipknts with 
those of a aomparable group of youth who were eligible. to participate but who . - 

did not.' While the time period since completion of the progi^ain m^y^b^ very " " 
brief at the time of the initial survey in 1979, the longitudinal nature, of ^ ^^^^5^^- 
the study will permit estimates of the gair^ from participation over time. A 
'wide variety of^ependent variables will be examined, including earnings,, 
welfare and other -^aniffer payment receipts, weeks' of employment vfiut^gl uiiempldy- , 
ment, aspirations, Job satisfaction, quality of working conditions,' length* \^ 
• of schooling, on~the-;;Job training received, and Job~sear,ch activities.. 

Factors in 'Educational Programs . The proposed Souciy will yiej,d^^ detailed . 
information on the pi^ogress -cr^^youth in high schpoi and college whi\:h can be 
used to provide answers to a number of policy-related 4'^estions' concerning 
botk tiie ckuses'and consequences of dropping put of school:, 

(1) WTiy do some school students complete thcLr education ^hile 

. 'Others i^ithdraw withou1?v4g;fetaining ^diploma? Wh.at is the relative importance 
of such factors as dif f ^i:er\ces in ability, differences in motivatibn, and 
differences in the economic status of the youth and their families? 

^ y ^ ' , , ' 

(2) Are high school dropouts at a disadvantroge compared v^th^gh school 
graduates in terms of esLrn^ngs and occupational status In their first Jobs? * 
If so, do these differences narrow and/or disappe'&r over time?, or do they V 

"♦ persist? ^ ; \ , . . 

(3) Is there any' eVidence-^that " private and public employe:^ use the high 
school diploma as a screening measure to classify app:*lcants* potential produc- 
t:|.vity on^the Job? ^ ' . ^ ^ ^ - . 

f 

Transition from School to Work . F:t^ a. policy perspective, the high • ' 

unemployment abates among youth during and^-in^the' ceveral years following the 
termination of scjhooling and the high rates of mobility during the early post- 
school yesirs suggest the need for a better understanding o'f the factors associ- 
-.at^d w4?th relatively smooth and rapid entry into satisfactory and satisfying 
employment situations. For example, among the questi?^? that may be addressed * 
are the following: ^ ' ' , \ ' 

^(l) What relation^ip is there, if an^, betve^.^ i Xtic high school experience,. ^ -iS'" 
including' work act^ivity, and the post-^school 'labor ■ activity of noEh-'' ^ 

colsl^ge bound youth? Do youngster^^ith work exp ^r! '*^".icc' do better, o^^^ - , • 

things equal, after they leava^p^rliool? D^es «the ^irj..wer to this questiSnSi^^ 
depen^on the extent and/or t?^e^of work experience? ' ' * ^^"^ 
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(2) Is there evidence of a "dual labor market" for noncollege bound youth, 
such that individuals with certain socioeconomic characteristics are li^tely 

to enter and to be trapped inflow paying and demeaning jobs in \rtiich traditipnal 
human capital variables appear to be irrelevant; or does the laboT market 
operate so as to sort individuals out among jobs pretty equitably in terms of 
their productive capabilities? 

-\ 

(3) In the early post-school years, what are the processes of mobility, 
post -school training, work experience, and modification of goals that-- result 
in youth settling into career jobs? 

£k) Does unsatisfactory experience (e.g., extensive 'unemployment ) in 
the imiaediate post-school years 'leave "scars" that affect later market 
behavior and experi-ence, or are these problems essentially transitory, with 
no lasting effects? " " , • 

(5) Are there various avenues of advancement for youth during their 
early exposure to the labor market?' 

(6) What IS the character of the extensive job changes made by youth? 
Does "job hopping" result in progressively better jobs along all of the 
dimensions described above, or does it simply represent a string of equally 
poor employment opportunities? -How do,es the answer to this question vary ^ 
depending on sex, race, ethnicity, and the socioeconomic and psychological 
characteristics of the youth? What is the effect of variation in the 
economic environment? 

(7) What kinds of jobs do youth consider desirable, and at what wage 
rates? What differences are there in this regarO. between those still in 
stihool and those who have ended their formal education? What is the extent 

of variation in this regard by sex, race, ethnicity, and socioeconomic status? 

Racial, Sex, and Ethnjc Differences in Employment and Earnings , One 
of the principal purposes of the NLS study of youth will be to efxamine racial, 
sex and' ethnic differences in employment and earnings. At a descriptive level, 
gross differences in employment arid earnings among various face, sex, and ethnic 
groups will be identified. In addition,", multivariate analysis .will be used 
to ascertain the underlying factors responsible for these gross differences. 
Differences among various race, sex, .and ethnic groups in *term^ of their human 
capital attributes are one source of the gross differences in earnings, A 
"^second source of gross, earnings differences is labor market discrimination,^ 
Using the technique of wage-gap analysis, it is possible to decompose the grdss 
difference in earnings between t^ groups (e.g., white males 'and black malea) 
into" a portion attributable to ■arnJ^erentiai^.endowments of those characteristics 
presumed to determine earnings and a* portion which reflects labor market 
discrimination, . • • - ^ 

Causes and Consequences of Marriage, Divorce and Fertility Among Youths 
In recent years nearly 20 ^percent of 'all births were to teenage mothers, ' <• 
Nearly hO percent of these births were illegitimate and nearly two-thirds of 
the mothers had not compJ.eted high school. Past research suggests that these 
women have much poorer prospects than do those who have children later — teenage 
mothers receive less education; h^e mor^ children, and have both a higher risk 
^6f divorce' and of becoming dependent on public assistance. For young men as 
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well as young women, early parenthood may curtail the amount of education they 
receive and reduce their earnings potential. The proposed extension of the NLS 
will make it possible to study a variety of related issues including: 

(1) What are the cultural, familial, attitudinal, and economic factors 
that increase the chances of early childbearirig, early marriage, and separation 
or divorce? How have these causal relationships changed over time? 

(2) What axe the social and^^gconomic consequences of early childbearing , 
marrieige and divorce? How do these effects vary according to sex, race, 
ethnicity, and socioeconomic status? 

(3) For youth who assume the, Iresponsibil^ties of marriage and child- 
bearing at early ages, and for young mothers whose marriages dissolve, what 
kinds of public interventions are most likely to be effective in promoting 
economic independence? What are the potential roles of (a) provision of child 
care, (b) counseling, (c)*acce^s to continuing education, and (d) job^training? . 

The Mobility of Youth . The NI5 provides a unique opportunity to examine 
in detail the associations betweeh geographic mobility^ local and national levels 
of economic activity, and social, economic, and demographic characteristics 
of youth.^and their families. The longitudinal survey' design, in conjunction 
with the plan to follow respondents regardless where they move (within the 
United States), will enable researchers to model the determinants and conse- 
quences of geographic mov-ement. The proposed study also will permit 
quantification of many other gross changes in aspects of the labor force 
status of youth, including movement into and out of the labor force and betweeh 
.employment and unemployment, movements between jobs, and between full- and 
part-time employment. Moreover, t'he relationship between these changes and 
\ changes in school enrollment status, demagraphic events, and work attitudes can 
be analyzed. Examination of changes in labor force and employment status in 
relationship to changing levels of national and local unemployment will permit 
the testing of the "discouraged worker" and the "additional workerV hypotheses 
and an analysis of a vsiriety of dimensions of frictional and "disguised" • 
unemployment\ 

Project Design • , ' ' ' - 

' ^ " * ^ _ *' 

. The sample vill* inplude 6,000 ypung men *and 6,000 young women between * 
the ages of ih and-21 as of January 1, 1979. -Therfe \7ili be 2,000 Hispanics, 
3>000 blacks, 2,00Q non-Hispanic, nonbla*ck youth from poor families, and 
•5>000 non-Hispanic, noriblacfc, youth f^om a cross-section of families. -Thus- / 
there will be heavy olfer-representation of youth who 'experience labor market - 
difficulties and who participate in the Departirient of Labor's anployment and 
training, prpgrams. These youth will be interviewed ajjnually beginning in 
January 1979 und continuing through the beginning of 198^1, for a" total of si^ 
interviews. Each interview will include approximately one hour of questioning 
and will usually bcc*ur in the youth's home. Overall responsibility for 
designing the surveys, analyzing the results, and making the data av§,ilable 
to the research "^community rests with the Center for Human Resource Research 
at The Ohio State University. The -National CJpinion Research Center at the 
University of Chicago has-been subcontracted to design the sample, condiict.the 
field work, and -prfijiare 'the initial data tapes and documentation. 

Interviewing will occur in 200 primary sampling units throughout the 
country selected so that the entire sample and each of its four subgroups will 
yi^ld nationally representative -;ta«tistics. In all, interviewing will "t)c cur 
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in 39 states and. the District of Columbia. Screening interviews in a repre- 
sentative sample of approximately 90,000 households are currently being 
conducted. in order to locate 'the individuals vho meet the age, race, ethnic 
background, and income standards which have been prescribed for inclusion . 
in the sample. 

Time Table , - , 

The proposed time table for the 1979 interview and analysis follows. 
Approximately the same schedule will occur annually for each .of the succeeding 
years, "Vfith the submis^on of the sixth fineLL r^ort being due on approx- t 
.imately September 1, 1985. 

First Year Activity ' '/ * ^ Date' 

Sampling design complete April 1, 1978 

Screening pretest I • June 1 June 30, 1978 

Screeping data collection' ♦ ^ n September '5 - December 5, 1978 

Respondents selected for full s^urvey January 2, 1979 

Annual Activities Date • • • ' , 

Interview .pretest ' ' ' August 5 - September 7, 1978 

Interviews ^ ^ January 31 - May 15-, 1979 

Data reduction ' * - ^ February 1 - August 31, 1979 

Data analysis b.egins ' \ September 1, 1979 
Initial report on survey (tabular analysis) December 31, 1979 

Final report on suirvey^ (multivariate - September 1, 198O 
^analysis) 

Potential Limitat-ipns * 

Any large scale .data collection effort faces a number of problems. 
Perhaps the most difficult of these is, imposed by the limftatrion of f\ands and ' 
time, given the large number of jiypotheses that would be desirable, to test and 

Jihe larg^-^ount of information that^^one woi:p4 ideally like to collect. When 
the' budget permits only ^ hour interview, the researcher is faced with the 
problem of whether to touch a number gf issues briefly or to concentrate^ on a 
few main tbp^ics and gather intensive infdrmatio^;i. .The longitudinal nature of 
the five-year study will . definitely provide the opportunity to ask more questions 
than can be used in the usual . government study .^^^ At. the game time, the longitudinal 
nature of the study requires that some information be collected year after* • ' 
year in order to make intertemporal 'compariaons , and this reduces the potential 
for' introducing new topics each ydar. We are hopeful that approximately half 
of 'each interview will' contain new infoi*Mation. *Hpwever, as we have learned . . 
from our recent efforts to reduce our preliminary questionnaire. by one-half much 

'important 'data must be foregone. 

Another limitation is^that our sa;mple will grow older over the period of 
Study and by the end *of the six-year period will be 20-27 years Oldi >they will 
no longer be "youth," SUch an aging"^ of the sample is obviously useful if one 
is to attemt)t to measure the longer-run consequences of employment and training 
program participation, education; training, etc. ' However, .it does have ^ the 
limitation that, what is true for 16 year-olds in 1979 may not be true in 1981^. 



-This limitation mity be very important because recent estimates by the 
Bureau of ' Labor Statistics indicate that the' proportion of youth in the labor 
force will fall substantially during the early 1980's and this 'drop in absolute 
numbers of youth may reduce the labor market problems faced by this group* The 
present study design will not shed any light on this issue however* This 
shortcoming in the study could be met by adding younge r persons to the sample 
on a-xegular-basi^ For rns1rance7 a new group of 1^ and 15 year-olds might be 
added in I98I and I983, and their experience compared with the 1^+ and 15 year 
olds in the 1975 group* < The regular addition of youth to the survey ^'should 
be given consideration by the , Department of Labor. 
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NATIONAL COUNCIL .ON EMPLOYMENT. POLICY YOUTH EVALUATION PROJECT ^ 

An evaluation of prime sponsor formula funded activities 

untter the Youth ^^ornmunity-Conservatrorr and -Improvement— Projects - — 

and the Youth Employment and Training Program^ 

The National CounciVon Employment Policy's Youth Evaluation Project 
is being done to provide the Office of Youth Programs with (1) a running 
account of the first year experience that prime sponsors are having in 
planning for, designing, implem^ntir^g and operating YCCIP and YETP, (2). an 
analysis of the factors imped^ Ibcal implementation of the new youth policy, 
and (3) recommendations for improving future operations. 



jestions for Examination 

The .evaluation is geared to assess prime sponsor experience with ' 
respect to certain issues embodied in the ten principles that are outlined 
in 'the Office of Youth Programs* August 1977 Planning Charter. 

'■ - ' \ 

YEDPA^was established largely as an extensive learning "knowledge 
development" --'experience. The Council's study is attempting to determine 
what "knowledge development" translates into at the local level: How do 
prime sponsors interpret the mandate, and what are they doing under the 
rubric of knowledge development? Has the mandate encouraged them to under- 
take fnnovat^ve approaches they might not have takea otherwise? How are the 
lesson-s gained from knowledge development experience uSed? 

An issue that historically has beeri at the heart of all youth programs 
is-' the quality of work experience. We^are trying to determine whether local 
sponsors are taking special steps to adapt newly created jobs to the special 
needs of y ou th s7"t o~ pro viae^ ef f ec tTve " s upe r vi g'l'oh , and tl) pToduce tangtbVe • 
output. ^ ^ ^ ' ' 
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The law and regulations /equire youth participation in the planning ♦ ' 
process; the Office of Youth Programs, vigorously encourages youth participation 
in the impJlefnentation and:management process". The evaluation is examining 
the role that youths are actually taking under the new^ policy, and assessing ^ 
the consequences'of that role. The evaluation is also identifying promising * ^ 
alternative roles of involvement. \ ' ^ ^ 

Although each program of YEDPA has its own target group and eligibility 
requirements, thfire is considerable leeway for local operators. They can 
either "cream" .those youths most responsive and likely to succeed or they can. 
attempt to serve-those youths less likely to be aided by more traditional' ' , 
manpower and education programs. The evaluation is assessing who ^in* particu^lar 
is selected for YCCIP and YETP and how local targeting pcflicies are implemented. 
The evaluation is also examining the practicaT implications and apparent 
effects of the variations in eligibility and targeting requirements and * 
practices among .the- different youth programs. 
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■As' with any new program that overlaps existing programs, the apparent. 
\effects of YEDPA may be diminished if local decisionmakers substitute new 
^resources jfor resources that would otherwise have been committed for programs 

helping vduths> Prime sponsor s_niay-.dQ_this^v u^ing-Y-ETP-money -in -l-i^u~of- 

CETA Title I money to provide services to youth. Local education agencies 
nay do>t by substituting YETP money for school resources that would otherwise ' 
be SV^filable to' provide services to which youths, by Uw, may^be entitled. 
The evaluation is assessing in qualitative terms whether the more obvious 
kfnds of substitution are- occurring. We. are also identifying some of the 
less obvious kinds of substitution that are occurring and diminishing the 

effects of YEDPA. . • " ^ ' 

J 

Besides presenting some , opportunities' for programmatic change, 
Y4DPA is presenting opportunities, for institutional change, both in administrative 
structures for service delivery and in the relationship^ between the education* 
establishment and employment programs. ''The evaluation "is gauging the impact 
of YETP and YCCIP' in stimulating ifisti tutidnal change at the local level, and 
analyzing the implications of those changes, , ' 

'""^ For prime sponsors implementing new prog rafnnia tic appV^oaches, tKer^ 

are conflicting pressures to try entirely new J)rogram organizations on one 
hand, or to stick wit'h- proven performers on the.other. . The choice can have 
implications for how well narrow program objectives are achieved, and for how 
well broad institutional objectives' "&re achieved. The .evaluation is studying 

* how prime sponsors choose their delivery agents, and how the performance of 
different .agents can be differentiated, ^ - , ' 

Finally, although YEDPA is part of a decentralized-decategorized 
approach to^ormulatjng and implementing emplbyment "and trai/iing pol\cy, it 

Reflects cerVin federal prioritises. As such, therejs an interest io How 
. faithfully the federal emphases are transmitted to local decisionmakers and 
how well they ar^e implemented.^ Responding to this interest, the Couhc-il's 
evaluation is also examining certain systemic, features of YETP and YCCIP 

implementation, and -is identifying the barriers that prevent or impede the 
real ization- of national policies. Jn the same vein, we are assessing local 
eicperience with respect to reasonable expectations about what YEDPA could 
acco/npl.ish .(based on unique local conditions), as well as with respect to 
broad patipnal objectives of^^'EDPA. 

K ^ Project Design 

A case study approach has be^n adopted for the evaluation because 
it was felt that such an approach is best adapted to Analyzing a new program 
that is both decategorized in its. prescriptions and-decentral ized in its 
decisionmaking patterns. The case study approach is sensitive. enCugh to . 
detect important^ variations in local strategies and tactics, and because it 
provides the flexibility to document and analyze issues that emerge oyer time. 



Ten case studies are 'being prepared by fie^d associates, each 
covering 3-5 prime sponsors. ^A total of 37 primje sponsors are covered by 
the case studies. 
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Jhe case studies actuStlly consist of three interim studies and a 
final study. A summary analysis accompanies e^ich set of 10 interim studies. . 
Tw(y interim report^^ have ^been completed; one in February and one iji August. ^ 
~- -A-th-ird will be submitted around'the first, of the year, and a final repart 
wilVbe submitted in June of 1979* 

The evaluation, so' far, is essentially exploratory in nature. We 
are emphasizing hypothesis formulation more than hypothesis testing. Because 
of that emphasis and the qualitative nature associated with it --^little 
systematic 'data has-been developed so far. Patterns in local progisalris and ^ 
managejnent styles make certain kinds of .tabulations j'mpossible or ii^practical . 
Some of the similarities are so ^el f.-ey'ident as to not warranl^much Analysis. 
The variations defy systematic quantitative analysis. , Given triese cit^cymstances, 
it would be pointless tcuattempt any elaborate quantitative analysis. 

Aside from the limitations in>ierent in our research design, thereyis 
a developing problem with respect to gaining access to knowledgeable program 
personnel. In some pUces, evaluators are encountering increasingly stiff ; 
resistance to their questions, and are being increasingly shunted to rigid* ; 
. bureaucratic channels for information. Some evaluators are finding^the welcome 
mat pulled in despite painstaking^ efforts to corroborate findings with 
different sources and desp-ite the practice of sol inciting the views of local 
administrators on case study drafts. * ' • ' 

Findings to Date and Policy Recommendations 
Baised on those Findings 

y^The experience of prime sponsors so far suggests, on balance a 

. ' significant record of success in implementing YEDPA and achieving its goals. 

Local -experiences also sugges^t some iT^easures that might be pursued at the 
\ I ' federal' and local levels in order to h'mprove performance even more. ^ 



(1) Planning for the fiscal year 1978 YETP and YCCIP activities Jagged 
behind schedule; it was rushed, confused, and marked by a disturbing duatism. 

Given the newness and complexity *of the youth programs, the requirements 
for review of local plans and the enormity of the implementation task, U is 
not surprising that the local .efforts fell'behind the Office of Youth Programs' 
ambitious timetable. 'With experience, the difficulties .encountered in the 
first year can be eliminated for most prime sponsors. Very large prime 
spoasors, with many buree^ucratic layers for review^, 'are certa'in to encounter 
chronic difficulties, however, j*n meeting evea^the loosest timetables*. 

The plamiing process itself consisted, almost inv^aria})ly , of two, 
independent procedures. There was/a pro formal process required, by the 
Department of Labor's gf;ant application package, and an ad hoc process'that 
was centered in the reaV local, decisionmaking arenas, and was based on data 
and other .factors distinct Trom those required by DOL. The DOL-mandated data 
were the ,kind of data associated with popular pjanni-ng paradigms, but wer;e not • 
used as a basis for. local deci^iomp^king, "they were deemed unreliable and 
too restrictive;* other variable? entered into the process. The grant 
'application package-also required data to be standardized in a format that 
was frequently not'*useful to locaVdecisionmakers. ^ - ; ' 
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. / The YCCIP/YETP -grant application process should be simplified to ^ * 
prbvide only information POL needs to support an .up or down decision on funding. , ^ 
If the Department of Labor wants to know more about local plans, it might ask 
fpr it on a selective basis, or require sponsors to document whatever planning 
p,i^ocess thexjjse^ ~* [ ^ ' ^ ^ ' ' 

/ (2) Knowledge^ Development , YEDPA marks a new emphasis on prilie 

sponsor involvement in evaluation, research, and demonstration activities. 
The stress on creating a new prime sponsor role in these areas naturally 
broiJi^ht with it a need for a, great deal of guidance. It also brought, with 
it the realization that in the first year accompl i^shment^s' wOuld be limited. 
On the positive side, it appears that nanV^prime sponsors ^_id^ increase 
evaluation , activities. - A minority tried out new approaches, directly in 
response to the call for knowledge development, A handftr^ atteoipted structured 
local experiments. It is almost certain that these types of activities were ^ 
far more prevalent than they would have been in the absence of the emphasis on / 
local knowledge development. These developments may help^ prepare a base for. 
absorbing the lessons of nationally directed discretionary. knowledge development 
activities. They may also lead to improvements locally. ' -^x 

It is doubtful, however, that local knowledge development ^activities 
will substantially improve overall understandi^ng of you^th problems and programs. 
There is uneven -capacity among prime sponsors to undertake this function. 
There continues -io be a good deal of confusion about the substance and form 
that knowledge development should take., The Department of Labor. provided too 
little guidance^about how prime sponsors might , approach knowledge development. 
In the face of the" ambiguous, open-ended knowledge developinent requirements, 
many prime sponsors di(i very little, or tried only to re^.ond to what local 
officials thought that DOL officials wanted. Others attempted, to set up 
excessively sophisticated research' projects that, by attejnpting^to do too 
much, probably w-ill accomplish v^ry little. * . 

The Department could remedy this situation by providing more tec)inica-K ^ \^ 

assi stance, directly to pr i me sponsors, or indirectly, by facilitating 
communicat ion amon g prime s ponsors . T,he Department shauTd also ass ure^tha t 
t herV is some minimum degree of agre e rent among t he- regional offices, with • 
respect to what knowledge deve>lopment is, and what is allowable in the name , 
of knowledge developments Further guid ance pn knowledge development should 
enco urage a Vink/ between what is bein^ labeled as "k nowl edge development,"" ^ 
and ot her evaluatio n/moni toring activities. ] ' ' • 

(3) The site reports suggest that supervision 'and job enrichment 
have been given beavy emphasis by CETA prime sponsors; to the extent t]iat= " .^y 
these factors affect the impact of work experience, the new youth programs 
have produced major improveinonts. YCCIP, which was designed as a well- ^ 
supervised work experience with limited e'nrichiTiOnt has been linked' with 
other programs t6 provide more than work experience alone. " . " 

Under YCCIP and YETP, prime sponsors^are giving much attention to ' 
career exploration for youth through actual work experience and through more 
structured occupational information channels. The atcent on placing youth 
in career-related jobs ^ is proving to be impractical.. Some youths have not 
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thought about career plans. Others^ have either, what mighl be considered 
unr:ealisttc plans, or plans fo\ jobs that'rfo nat fit in with the youth joi)s 
tha-t are available. The emphasis on career-related jobs is also prematures " • 
for many youths, in the sense that- they have* only the vaguest notions c^bout' 
the world of *w(5rk: Before they can appreciate the differentiating details' 
of different kinds of jobs, they need lo understand the common recjuiremervts 
of all jobs.^. ^ . , ' V \ ^ . ^ . 

t)n* the other hand, vocationa^l exploration classes and counseling ^ . 
seem to be more effective mecha^ni sm.s. for providing ^exposure to» dif fer.ent ^ 
career option^. To the' extent that care er explora tion is emptiasized, prime 
Qpn^r^<;hnindjTig_^^ through c ounsel i ng and Clas sroom ' ^. 

PV£^7ipT^^^ devVioprnent -Campaign should be 

on jToHc exp erience that teachVs, at a' minimum , ba sic work habits, and the ' 
importance of th ose ha bits > ' - 

--C«>*- , — — ' — ' - -— „ 5> , ^ 

' While prime sponsors have increased eiifphS'sis on {private setter * 

linkages, ajid while their efforts have had positive effects in few cases, * ' \ 
contihued emphasis is needed. Prime sp onsors should ,be encourax)ed ^''^more* 
than they are a1rea.dv, to attempt to establ tsV -a- ^trohg .bond with pr ivate 
sectoryinterests. -.More practically, the y shoul d be encouraged to evaliTate 
job slot develbpmint plans wit h reference to how much of what is learned 
, in- a certain job-is transferrabl e to private .se ctor em pl o.yfnent. » 

4 

(4) .Prime sponsors have followed the regulations reguiring youth 
'■membership ori" youth advisory council-s: However, the advisory council 

appears to bel a^ poor vehicfe for incf'easing youth participation in prime 
sponsor affair^.' The councils th^.mselves 'are often mere figureheads, and 
youths on- the coujici-ls are not participating very much. Some sponsors-are 
. experimenting .with alternati-ve modes pf parj^lci'pation, and. are enjoying 
^Ame degree of success. • ■ ' I .' ' * 

Al th ough Depar tmental^u^anco^jn .incLeasioq vouth participation _i_s- _ 
no t_lj£rted. to y outh advfsoT^" coujicfl s_^.hore s hou ld_ b e more emphasis on 
-p7i1ie]jj3qn^5qrs _ej<£er^i^^^^ .■(or'Ti^ ,oF^<5r6cijDati 

(5) The YED^ legislation provideSo pr im.v s.ponsor!; with a degree of 
latitude in choosing who to, enroll in youch programs iiowever, the Dnployment 
and Training Admi nistration ' has gonq, boyond che. legi slation in emphasizing , 
targeting on youth most in. need. Prime sponsors have' responded positively , 

by consistently, aiming services for" economically di sady.antaged youths. There 
-^ay be some force of'habi-t in effect,' but local pre^^sures for serving" those 
-most in need are a Isd^ persuasive . Enrolljwits s_hou1d._bj2 -mqi^^^^ 
whether prime _spjDnsqrs^5DtLnue_to^ on.'iervj-"^ economically 

d~isad"v'anfag¥d'XoulL. Jl/when there is. a jhift In tar5eUna2.Xuj:tber,m^^^ 
shoujd be" unde£t\Ken to deYe7m>ne_y^^^^ theje j_s_a_reasc^ " 
• suFh siii/ts . ' 

* Prime sponsors are seriously^undorcnrsll ing females in YCCIP 
projects. It is not clear whether this is duo to* relatively few' females in • 
the YCCIP applicant pool or to discrimination in job placements. Ln„e_Ui]ei. 
case^u^rim e spon sors s_hould be encouraged to undertake more outreach activity, 
soT hat female .enrollm ents'. in YCClT^Qan -bfe i^aiied^^ ; 
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(6) /.Substftutitin of-YEDPA resources for Title. I resources is not / 
proving to be a. serious problem. ^ In .fact» there has been a remarkably degree ^ 
.af -utilization of 'other services 'ahd 'programs 'to enVicb' the. yo.uth Initiatives. ^ ' " 
■ However, .substitution of YEDPA resources for non-QETA- resources could prove 
to seriously diminish the impact of the new funds. ' There is Tittle .that> can 
be done,* legal ly^ -to porrtrol'that kind of . Subsiitutioa. But where -le.gal 
recourse is limited, moral suasion' might be 'more effective. . • / ^ ^ 

• > . ' : : ' ^ ' * - ^ . ^ 

'^O ne way to increase the interest>$ of loca l sponsors a nd^ndn-CETA 
insti tutio ns in minimizing substitu ti on is to g ive them a sta^e -in the actio». 
This, unfortuhatel>^, is easier. saTd than -done; SponFsors are already being 
encouraged to bui>d-link^ w-ith" other 'agencies Bnd private sector interests. 
'l^jLlh.t^.'-.^Mi^^^ P_.y.s^^djjeyen^ mre, o_n jth?: JJQu.lds .that A broader 1 ocal ' - 
base of-lpartjciijatTon^ will better assTjre a broad'e^r rcoristi tuency "intereste d 
in s tretching YEDPA r e sources to the maximum . It would be naive^to assume * ' 
that a coherent sense of purpose among man^ players could be developed, or : 
fhat this-alone wfll prevent substitution. But since the substitution 
'phenomenon is so nebu'lous. arid difficult to control at^times, mo.re effective ^ . 
solutions may not be feasible. ^ * c. • 

" -« * 

• •__j7l_XiiQ5Punity based Organizations are thoroughly involved in the 
youth program delivery matrix. Although performance differentiates are 'hard 
to^^d^tect, the GBOs do'broaden "the base of local participation. T he current * ^ 

p rovis ions giving them special consideration in the s electio^n o f program^ 
deli very agents, and including them in the pJanning proce s s s hould be reta i ned . 

^ ^ ^ • - "-^'^ ./ 

(8)/YEDPA_..sM^ promise for altering institutional roles and-'. . 

• relationships at the local level,. The role of local- schools is central to . 
this-effort. In spite of the rushed implementation,'' CETA-LEA a-greements ,h?ive . 
been negotiated arid programs are in place. Even in areas-with little fiis'tory 
of-CETA-LEA cooperation, the separate CETA and LEA communities are talking " - - 
with -one. another, and j.oint efforts are underway. Nevertheless, there are " * 
Srome obyious^.li^i tafions on^^ow-.much change' the^Dep^irtment of Labor can. . - 
leverage in LEAs, by using prime sponsors. LEAs are--'not accustomect to.' 

. receiving guidance from EETA prime sponsors . Under the hest,J3f conditions, • ^' 
prime spohsors can hope/^o affect local educatio^i policy only margjn^lly. 
.In 5om6 of the worst geses, LfAs.and sponsors do not speak at all, . ^ 

. / • . . ' . . 

• ' Fo r- these reasons, the Dep artment of Labo r should exj )l ore .-"alternate 
ch'd'rine^l s "for af fegting education degisionmaking at thejocc^l 1 evel . U n tj* 1 ^ ^. 
those otfier channels for conmunication are Identified and uti n'zed.,'maoy 
schopls will fall to meet the needs of. youths who could otherwise be' served 
under cooperative arrangements with LEAs and prime .sponsors, or even through 
the LEAs alone\ I ^\ ^ ^ * 

On^/of the significant patterns that seems to be emerging is;;xhe • 
(iresence of/LEAs where CETA-private sector links exist. It is possiMe th^t 
LEAs may provide the missing piece necessary to complete ^productive relationships 
between pwblic sector CETA dgcmcies and pri\7ate sector employers. The Department 
of Xabor should e xamine C£TA a cti vitie s that invol ve ^private sector t^ies, s¥e 
v/hether^ LtAs_ pTa y aln integral r ol e. The ^Department sho\ild? lso stud y, some o f 
i ts exemplary CEtA-LEA projects t o 'eva-luate their private sector links where 
t hey exis.t, and to estimate' the f easibility Q.f such Itnks whe r e they do npt 
exist. s ' . J ' ' ' * 
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^' . ^ Project Overview Statement 1/ • * * 

^ . ' ' IjcoTiomic Impact of Jo b gorp^ 

' • by Seymour Drandwein 

Director , '"Of f ice of Program Evaluation 
Employment and Training' Adijninistration 

Unlike the YEDPA programs focused on by most of this 
conference, the Job Corps has been in operation for some 
time. It has survived over 14 years of fluctuating atti- 
. tudes towards sizaj^le investments' to aid disadvantaged 
youth. 

' F'z^ortf turbulent' start-up years, it has 'evolved into 
^ a relatively .stable program with a good notion of what 
it doing, toward its general statutorV purpose (first 
• stated in th'(? -1964 Economic Opportunity Act and since 
carried forward in CETA) "to assist young persons who p 
need and .can benefit, from ani unusually intensive' program, 
operated in a group' setting, to become more responsive, 
employable, and productive citizens..." 

Because dts intensive nature and ^izable investment 
per* participant should yield reasonably measurable re- 
sults, because little was known^ reliably about 'ti^ mag- 
nitude or timing of economic effects of such investments, 
and because the evaluation art had advanced somewhat 
since the Job Corps wa^ the subject of controversial 
evaluations^ in 'the l&te 1960s, we decided in 1976 to 
' undertake evaluation of the Jq^d Corps- as the principal 
impact evaluation effort rn the youth field: 

A competitive procurement ended up selecting . 
Mathematica Policy Research ,to conduct' the evaluation^ 
The preliminary findings on the short-term economic 
effects of the 1977 Job Corps program arc now in and 
being^ reviewed, with a report sehcduloc^ to be issued in 
November . ' • * ' 



1/ Prepared for "Knowledge Development Projects 
Conference" of Office of Youth Programs, 
October 5-6, >978. . . 
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' Objectives ^of the Evaluation ; The principal purposes 
of the evaluation are? to determine: 



1. To what extent do Job- Corps participants (as 
compared to comparat)le nonparticipants) : 

'(a) increase their employment and earnings, 

'(b) ' return to school, enter military service, 
o'r enter other training or work experience 
programs • ^ 

(c) rely less on -welfare transfer-payment 
programs. 

(d) reduce antisocial behavipr^v particularly 
^ \y criminal activities and vdrug abuse. 



2.^ To what exU^nt dp such effects vary by type of 
participant, duration of participation in the program, 
> and by type of Job Corps center? * » ' - ' 

3:- Iiov> do the Job Corps participan£s rate their satis 
faction with the program? What is their assessment of 
\ its strengths and weaknesses? 

4. Do. the program benefits -to participants and tp 
society where measurable in dollar * terms , outweigh 
the program cost^? 

Design ; In brief , the program impact is measured 
primarily by comparing the experience of Job Corps parti- 
cipants over time ?o that of a group of similaif youth not 
enrolled in the Job Corps, 

The comparison- group was drawn from ;school dropout 
lists and Employment Service applicant files in geographic 
areas which had relatively low Job Corps participation 
rates (in large paint because they were not near Jojp Corps 
centers) , but which^Ln other respects were similar to 
areas from v/hich most Job Corps enroilees have come. 

The baseline interviews OJE participants^ were con- 
ducted at Job Corps centers in Spring 197,7 tb obtain in- 
formation on demographic characteristics, socioeconomic 
background, employment and earnings, education and 
training,' antisocial t)ehavior , and participants, assess- 
m.er^t of the program: 
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* The' sample, some>5,100, was a random selection of ' , 
'about one- third of participants. Nine months later, 1 
thqse who had terminated and been out at least 5 months 
were interviewed again to obtain information on their 
early -^ost^rbgram experience; the followup sample was 
about 2,400, who had been out an average of .7 'months. 

Tjie comparison group baseline sample was 1, 500. 
The .followup interviews were obtained , for some 1,^00 of • 
. them. ^ ' ^ ^ 

A baseline data analysis of the participant: and • 
comparison groups' characteristics and prior history in- 
dicates thajt. the two- groups were quit? similar in fehese 
respects. , . r * ' ^ 

I^n addition to the interview information, 'the study 
drew cost data from Job Corps operational- records' and 
developed estimates of the value^of output ^produced by 
participants as part of their training and wprk while in 
the program. 

-* 

Preliminary Findings ? An early report on the baseline 
interview data, "An Examination of Job Corps Participation, 
was publi^shed in June 1978, and is available on reques,t to 
the Of fici' of Program Evaluation. 

< . ' — ^ ^ \ 

It describes the nature of the Job Corps program in 
1977 , ^ documents the "disadvantaged" nat-Are of those it' 
' is serving (enrollee demographics, family background and 
^ income, prior education^ health, and work histor^^s , ^.and 
"antisocial behavior"), r'ev^ws why they' chose to enroll^ 
and presents their ratings of 'the program while they were 
still in it (alm©^ two-thirds were "fai^rly" or "very"^ 
satisfied, over 90 percent thought their tVair\ing ^as 
*^good" or "OK," bat half were , critical of , the fpod and 
size^ of' their pay allowances) . . 

^ ^ The followup data has now beep^eveloped and analysed 
in a preliminary ^report which' is^eing ^given tecihnical . 
rev-iew by an outside expert advisory grou^) and ^ by Labqr 
* Department staff/ * ' - ' * 

' -Its tentative findings on short-tgrms impacts^ on 
. labor market experience are: 

1. Program completer* (about 30 percent of all^ * 
enrollees) did very well ot\^ employment ^and earniiig^, 



as coippared to the comparison group, for both 
iqen^nd women, but, "partial t:omplet6rs" '(}0^ ^ 
pe^rcem^) and' early dropouts (40 p'ercent) did 
^ only about as well as the comparison group. 

2. In the initial months out of the Job Corps, 
its participants lagged behind the comparison 
group, many of who were -employed at the start of 
the period while^it took time for the. Corpsmembers 
' . to resettle and get jobs. 

-r- 3. After the first two months, the positive ^ 
economic impacts began to predominate for the 
completers. By.thfe week before the followup 
survey,, the completers (both male and female) had ^ 
, gained about $23 more in weekly earnings than. the 

' comparison group. The partial completers were 

slightly' ahead and the early dropouts were behind, ' 
although the differences were generally » statistically 
insignificant. 

On\Dther measures of impact too, the completers had 
positive, large, and statistically significant benefits: ' 
more were in the labor force, more wer^ employed, they 
averaged/ more hours at work, more of the men were in dfy 
military service (a goal of some Job Corps par ticipantsf 4 
more had gone on to a high, school degree college or more" 
training, and th-ey had less welfare dependence and re- 
duced arrests and abuse of drugs. For those who partially 
complected the program, such net impacts were small, and 
*--tlrT>^outs did not do as well as the comparison group. ' ^ 

^ As vto enrollee ratings of the program , 7 monJths 
after leaving it> 77 percent. expressed satisfaction - 
(versus 67 percent giving, such ratings while they were v 
still in the prog'ram) . 

Some 90 percent or ipore gave good or OK xating^ to 
the jbraining and education services they received. I In 
retrospect, they rated the food and pay better than they 
did whil© in the prografh (b^ut with a third* to ndarly" half 
still critical of those two aspects) . ^ 

^ Tbey felt^ they did not get enough placement^ assis- 
tance. Nearly) ^60 percent ^id that they did^not get 
placement aid^ from the Job corps or acjcricies the* Job . 
Corps sent them to and* three-quarters of thciso said 
they^ could have uspd more help in findrng a. job. 
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On the relation of benefits to costs , the study 
draft report develops some comprehensive estimates of the 
dollar value of various measurable benefits t_o. participants 
and to society as a whole. 

These tentative estimates depend-^ very substantially 
on various assumptions, most notably about continuation 
rates of gains in subsequent years, about appropriate 
rates of discounting future dollars to present values, and 
about* the* reliability and relevance Qf data on societal 
savings from reductions in criminal activity. 

The initial estimate is that economic "beneii.ts" to 
society in dollar terms for eac-h* Job Corps enrollee'are 
larger than* the economic "costs'^Tof the program; societal 
benefits per Corpsmember are 'estimated at $5,270, almost 
$200 mor^ than' the costs. . 

-.-^ * 

There patently, are considerable issues on the cpnceptual 
and' technical judgmentsuriderlying these estimates (as well 
as issues of their significance for policy on scale o-r mix 
of youth program approaches)*, but these are beyond the scope 
of this brief paper. / * , . 

^ Plans : Issuance of a report *i^ planned for November 
1978, after .the. technical review now being made of analytic 
assumptions and methods*. No doubt it will generate further • 
technical comment^ which would be taken -into adcount. in 
planned continuation of this evaluation ♦ ^ ^ 

. " The *key limitation of the study is that its findings . 
are based on^ just the first 7 months of ppstprogram ex-: 
perience. It is by no means clea:jii^ whether the partipipants I 
ppsitive gains, in this. early period are maintained, de- 
teriorate* (as the ben^fit-ccrt calculations- assume) , or' 
increase further.. ' ° ^ ^ 

♦ The evaluation is therefofe to be carried forward 
(as was originally planned) to .follow up on the partici- 
pant and comparison groups in ^similar intcrvit^ws <\ yoaa\ - 
after the first foTlowup, by which time the first folJow- 
up sample will have been out of the program for -an average 
x>f some 19 months. ^ . i"' ' 

We plan, for budgetary reasons and because of indi- 
cations that it may be as, -efficient, for repeated inter-- 
viewing, to' conduct the second followup on a telephone 
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rather than' facer'to-f ace basis , v. although" personal'' inter-, 
views will be\so.ugh't where the individual is no.t locatable 
or responsiv.e by phone;. ~~ . 

• • * * ' 

V (Data will also -be collected on the baseline inter- 
viewees who had not yet left the program or ha-d been 'out 
less than S^mori^hs at the, time of .the 'first followup- 
This will enlarge the sample of terminees and enable sqme 
check on^whether the findi-ngs hold* up. for participants 
who left the program at a "later point J.n"tiraeL,_ __ , _ 

Findings 'froxn this second fol^lowup shoul^ be ready 
by la'te 1979. ^ Still another' followup at a l^Cer. point is' 
a possibility , but- that, is -contingent- in major part on 
likely a'dequacy .of the sample in light .of 'response rate*,* 
experience-;in the second followup.^ ^ . * • \ 
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